69 replies to this topic

[TomH]

After some cleanup on the TMG side, I did a fresh import into RM.  In TMG I have the following sentence template:  [D] he registered for the World War I draft [L]. <[M]>

 

Upon import, the resulting RootsMagic sentence template is:  [C=M,0  ,0  ][Date] he registered for the World War I draft [Place]. <[Desc]>.

And the resulting sentence in a narrative report: [C=M,0  ,0]on 11 Sep 1918 he registered for the World War I draft in Winder, Barrow County, Georgia.

 

This would be perfect if not for that odd little [C=M,0  ,0].  This isn't the only sentence template where I'm seeing this, but it's probably the simplest one where it's happening.

I have not seen this in my limited testing . Do you have any idea what it might represent in TMG?

[TomH]

Another problem I'm seeing is in sources.  A lot of my source citation templates use an optional citation memo (in TMGspeak, <[CM]>). 

 

In endnotes for narrative reports in RM, these are showing up as [Cm].

 

As far as I can see in the construction of RM sentence templates, <> means the same thing as in TMG - use only if present.  Maybe it's different for source templates???

TMG's "<[CM]>" is carried over without translation to the RootsMagic sentence template as <[Cm]>. The TMG Citation Memo is imported into RM Citation Comments. Because RootsMagic does not support the embedding of Citation Comments or Research Notes in footnote sentences, it has no equivalent variables and cannot be translated. When a sentence template has a variable in it for which there is no field definition, the outputted sentence displays the variable name, in this case, [Cm]. That acts as an indicator that you need to do something about it. Either you delete the variable from the source template sentence templates or you define it as a field or replace it with another field definition and copy some content from the Citation Comments into the corresponding field space in the Citation Editor.

 

The Citation Comments can be outputted in reports with the footnote by the choice of an option in the report settings so it is not absolutely necessary to have an equivalent to [CM] to cause them to be. However, there is no control over which sources have comments printed and which do not as such a variable might provide.

 

I don't know if there is anything programmatic that can be done to mitigate the amount of editing this may entail. I have written SQLite scripts that post-process a direct import to handle the [CD] problem and transfer short event notes (TMG memos) to event Description. Perhaps there are some parallels. 

[CherylCh]

I have not seen this in my limited testing . Do you have any idea what it might represent in TMG?

I"m not sure, particularly as it isn't showing up consistently.  I don't know if you are familiar with TMG Utility, which is a companion program that does some of the direct database manipulation that you do with SQL queries.  I used TMG-U, and that odd little character string isn't itself present in my dataset.  I will poke around and see if I can find some commonality in those cases where this is happening.  But the dataset in question has 14,000+ people in it, so it may be hard to figure out what the issue is.

[CherylCh]

I'm seeing something similar with some witness roles.  One of the areas in which I have customized tags (facts) and roles is with respect to the census.  For each US census, I have a separate fact type and multiple roles.  For 1880 and later, the roles correspond to the various relationships specified in the census (wife, son, granddaughter, etc).  Seemingly at random, the entire sentence template in these cases is replaced in RM with [C=F,0 ,0 ].  In some cases, the TMG sentence template appears, translated into an RM template but preceded by the [C=M,0  ,0  ], as described above.

 

I'll see if I can identify any commonality in these instances, but I don't have any way to directly access the TMG database other than TMG Utility and even that's at one remove, so to speak.

[TomH]

It may well be then that these [C=...] things are not anything you enter as data but are created by TMG and mishandled by RM or are created within RM's import process. Thus one might surmise that it is a bug in the direct import process but it also might be due to some form of corruption in your TMG project database. Have you followed the recommendations:

 

Before Importing Your TMG Data

This section details cleanup

items you should perform in TMG before importing into

RootsMagic.

● Use TMG Utility to export any “internal” media items to “external” media items.

● The TMG project should be using "Custom" source categories (Preferences / Current

Project Options / Other).

● Import can uncover hidden problems with TMG sources. If a source doesn't import

correctly, the problem likely is the TMG source. For example, in TMG, the source type

and the fields on the Source Definition / General tab don't match. In this case, select a

different source type and then reselect the original source type.

● The repository NameOther

field is not imported. Make sure you copy that to the

Addressee field (if empty) or to the Memo field (if the Addressee field is used).

 

Importing Your TMG Data

● The direct TMG import has been tested with TMG versions 7.04 and 9.03. The

recommended TMG versions to use for import are 7.04, 8.08, or later.

● Before import, run the TMG maintenance routines optimize, validate file integrity, optimize.

● Make sure TMG files are not open in any program before trying to import them into RM.

● Use several import cycles. Import. Check data. Make edits in TMG to improve import.

Import again.

 

If repeating the import after repeating the above results in the same errors, then the likelihood is a bug. RootsMagic should want to receive your project file backup to try and replicate the problem and will want to be pointed to examples you have come across. You can submit a request through the Support page on the RootsMagic website or directly via http://support.roots...us_requests/new

Edited by TomH, 01 October 2014 - 08:31 PM.

[TomH]

Cheryl, I have concluded that this is an oversight in the RootsMagic Direct Import process. The TMG File Structures document p.21 says it is an optional consistency rule used for each language definition for the tag type sentence:

 

[C=AppropriateSex,MinimumAge,MaximumAge]

 

I'm not sure exactly what it does but I believe RootsMagic has no equivalent. It may be another case of an untranslatable being left exposed for the user to correct but, perhaps, it is preferable that it be deleted or skipped over in the import. The rules do not appear in either of the two files I have looked at and may not have been in any of the files that the RootsMagician and Jim Byram tested as it has not come up as an issue before now.

 

To correct it post factum, you might find them using Search > Find Everywhere. A sure way is running the Source List report, saving it to PDF, RTF, or Text and searching for "[C=" in the report. Perhaps you have means within TMG to search them out and decide if you can dispense with them there as they do not appear to be commonly used.

 

Edit: I see that the [C=M/F, x, y] rule is probably hidden from TMG search but is created when you edit a tag type in the section:

"This role is appropriate for: 

Sex

Minimum age

Maximum age" 

I think that only when any of these is changed from the default "Any" is the [C= expression actually inserted into the sentence template.

Edited by TomH, 01 October 2014 - 10:00 PM.

[Jim Byram]

My fault. Our focus was on what could be imported. I didn't create an example with the sex, start year, end year structure and I didn't see the issue with the user projects that I tested. The empty structure was encountered in v7.04 projects and Bruce added code to strip those.

[CherylCh]

I'm glad to know what that is.  I don't believe I've ever specified ages in tags, but gender, yes.  I have some custom census tags, with custom roles that are gender specific and based on the relationships reported in the 1880 Census and later.

 

I've been cleaning up my TMG database over the last couple of weeks, sort of like the "deferred maintenance done" when trying to sell a house. I did read and follow the suggestions for improving the TMG import, and as I am doing housekeeping, I'm reimporting and trying to figure out what is easier to fix on the TMG side and what might be easier to fix in RM.  I need to rewrite the sentences for those census tags in any case to clean up how I used place variables.  Should I go ahead and get rid of the gender specifications, or is it likely that the import routine will be changed to ignore those?  Should I go ahead and contact support as you suggested, or can I rest easy now that Mr. Byram is involved (he has been very helpful to me on the TMG forum)?

[Laura]

It means the same in source templates.

[CherylCh]

Thanks, Laura.  I just now realized which of my many questions you were answering!

[Sherman Peabody]

I have redone my import using version 6.3.3.2.  Some of the issues I listed in message #17 in this discussion appear to be fixed while others are still outstanding.  Tom, I appreciate your detailed reply (message #18) but apparently I am not explaining what I am seeing very well.  I will try to be more specific:

 

Two examples:

• I have a custom Milit-Reg tag which mimics the standard TMG Milit-Svc tag and sentences. I have a role of "NearestRelative" defined with the sentence "[S] <|and [P2]> was cited as the nearest relative of [P1] on his military registration <[D]> <[L]>".  The [P2] variable is referenced even though I never enter a second principal for this tag.  This role sentence is translated into "[Person] was cited as the nearest relative of [Person] on his military registration <[Date]> <[Place]>".  In this witness sentence [S] should be translated as [ThisPerson].


•  It is possible that an Event-Misc tag might have two principals, but I have an Event-Misc tag with only one principal entered.  I have a witness sentence customized for this tag that says, "[W] was an attendee when [P1] was a presenter at the Mid-south Training for Service <[D]> <[L]>". In RM this witness sentence is translated to "[ThisPerson] was an attendee when [Husband] was a presenter at the Mid-south Training for Service <[Date]> <[Place]>."  The resulting sentence displayed from this sentence template didn't know what to do with [Husband] so it is unevaluated.  The import created a "Miscellaneous" fact, but it appears it translated the sentence as if it was a "Miscellaneous (fam)" fact.

Hopefully this is clearer.

[Jim Byram]

Sherman,

 

There is no way to distinguish whether [S] applies to a principal or witness. [S] is assumed to refer to a principal for the purpose of the import so imports as [Person].

 

In your second example, [Husband] is a import glitch and should be [Person]. I found several of those doing my cleanup and fixed them. I just saw other examples on the REFUGEES mailing list.

 

Jim

Edited by Jim Byram, 09 October 2014 - 07:59 AM.

[CherylCh]

I did a fresh import using v.6.3.3.2 and noticed the following issues:

 

1.  A lot of my source templates include an optional citation memo <[CM]> that I use to explain such things as how the cited source applies to the fact in question.  An example would be an approximate marriage date with a census cited as the source, and the citation memo says "The marriage date is estimated based on the birthdate of the oldest known child."  Upon import into RM, [Cm] appears in all (I think) citations where <[CM]> appeared in TMG.  Any text that was in a TMG citation memo shows up in the research notes/comments field and is included with endnotes if "print detail comments" is selected in report options.  I'd really like for that [Cm] text to go away.

 

2. In sentences with roles limited to a single gender, the import shows [C=M,0, 0] or [C=F,0, 0] instead of or in addition to the actual sentence text for individuals with that role.

 

3.  In some of my sources, i have a File Reference field that I use to identify the specific location of the paper copy of a document in my filing system.  I use exclusion brackets in the Bibliography template {[FILE REFERENCE]} because I don't want this ever to print out as a footnote, but TMG requires that a field exist within a template in order to use that field.  It's just for my personal reference.  This information is being dropped entirely in the import process.  From my perspective as a newcomer to RM, the Secondary Repository field might work well for that, or even Source Comments.  As long as it's somewhere, I can move it if I need to.

[TomH]

1. Lists > Source Templates. Browse through the Source Templates prefixed "TMG" to find sentences bearing [Cm]. Edit the template to remove from the sentence and delete it from the Fields. That should remove it from all citations of sources built from that template.

2. This issue has been raised before; the import should ignore the gender control.

3. That's a new one to me. RootsMagic does not honour what it calls privacy braces in source sentences, something that needs to be corrected, but I thought it was importing them.

[CherylCh]

Tom,

 

I know I can edit the template and make [Cm] disappear, but since it doesn't function post import, and the information is going elsewhere, I'd rather the import made it disappear.  I don't suppose you have an SQL query for that???

 

Speaking of which, as I understand it, in order to use the SQLite tools you create, or to make my own, I need to get some software.  I'm running RM6 using Parallels on a Mac.   Do you know of any issues with using the SQLite managers listed on your website in a Parallels environment?  I'm under the impression that once a Mac version of RM is available, I'll then have to get an SQLite manager for the Mac side of the house - is that correct?

[koornalla]

On the subject of TMG migration issues I have had some rather interesting results with the migration of TMG Census facts into RM. In my particular situation the following occurred:

 

1. All TMG Census fact templates migrated across to RM complete with roles included (great).

2. The sentences for the roles within these facts did not migrate (not a big issue).

 

I was reasonably comfortable with the migration thus far but the issues start to appear with the migration of the TMG Census facts and how they are presented within RM. It appears that TMG facts with the standard wife, daughter and son roles within the Witness area of the TMG fact record migrate OK. I use the role "Head of Household" the Principle area of the TMG  fact record in almost every situation. I don't know whether the migration issue is associated with this field of the variation of roll type outside standard few. I feel that it is the TMG Principle field within the Census record is probably the culprid

 

In my situation, in the RM person record for the person who appeared in the Principle area of TMG as "Head of Household, RM generates an appropriate Census fact for this person It shows everyone in the household sharing the fact with this person. It also generates Census facts for all others that share this record. But it does not show this person as sharing the fact. If a was to go back several censuses for this person when he was a son in his parents household (in the witness area of the TMG record) he appears as a person sharing the record. I know that within the RM Census record generated RM sees this person as holding a "Principle role within this record because RM does generate a sentence for this person and it is a sentence associated with the RM principle roll.

 

My issue is twofold.

 

1. I cannot generate a "Head of Household" sentence for this person unless I change the Principle roll sentence within the Census template to reflect this.

1. I cannot ever change the roll of this person as this appears to have to be done within the fact shared area of this fact. This person does not appear in this area. This would be an abvious way to resolve the problem.

 

 

Feedback on this issue will be very welcome. I know this report is a bit long winded. Sorry!!

 

 

Wayne Thurley

[Jim Byram]

Wayne,

 

RM supports one global role for principals... Principal. If you want a particular global fact type role sentence for the Principal, you need to copy that sentence and replace the default principal sentence.

 

To put it a different way... You cannot use the 'Head of Household' role for the fact type role Principal. You can use the 'Head of Household' role sentence to replace the Principal role sentence.

 

The only way to have different principal sentences is to use the local sentence in a fact.

 

Jim

[TomH]

I know I can edit the template and make [Cm] disappear, but since it doesn't function post import, and the information is going elsewhere, I'd rather the import made it disappear.  I don't suppose you have an SQL query for that???

Not yet... The dilemma for the RootsMagician, I guess, is whether to leave or delete untranslatable fields. Leaving them raises the [LOOK AT ME] indicator so that you decide what to do about it. For example, some might decide to move something from the Citation Comments into a Citation field while you just want to be rid of the nuisance.

Speaking of which, as I understand it, in order to use the SQLite tools you create, or to make my own, I need to get some software.  I'm running RM6 using Parallels on a Mac.   Do you know of any issues with using the SQLite managers listed on your website in a Parallels environment?  I'm under the impression that once a Mac version of RM is available, I'll then have to get an SQLite manager for the Mac side of the house - is that correct?

I have no knowledge about running SQLite managers written for Windows in a Parallels environment. Yes, you will need a SQLite manager written for the Mac to work on the database in the Mac OS. However, if the database is visible to both Mac OS and to Parallels, perhaps you can continue with the Windows SQLite manager.

[CherylCh]

I'm seeing something kind of odd in my latest TMG > RM import.  I have customized witness roles for the Death tag as follows:

[R:Survivor] survived the death of his wife, [R:Deceased], <[D]>

[R:Survivor] survived the death of her husband, [R:Deceased], <[D]>

 

Let's say Harriet Smith died on 9 Oct 2014, and was survived by her husband, Harry Smith

 

On import, the resulting RM template is:  [Person] survived the death of his wife, [Deceased], <[Date]> .  The resulting sentence in the Edit Person screen and narrative reports is:

Harriet Smith survived the death of his wife, , on 9 Oct 2014 .

 

And of course, that's not what I intended.  What I want to see is "Harry Smith survived the death of his wife, Harriet Smith, on 9 Oct 2014.

 

Is something like this even possible in RM?  I set it up in TMG so that, particularly in the case of multiple marriages, I'd have a reminder in the Details window of when each marriage began and ended.  In the Timeline view in RM, the death date of a spouse is already there.  The presence of this sentence in narrative reports is somewhat less important to me, although it would be nice to have.

[Sherman Peabody]

Sherman,

 

There is no way to distinguish whether [S] applies to a principal or witness. [S] is assumed to refer to a principal for the purpose of the import so imports as [Person].

 

In your second example, [Husband] is a import glitch and should be [Person]. I found several of those doing my cleanup and fixed them. I just saw other examples on the REFUGEES mailing list.

 

Jim

Frankly, I don't recall customizing any TMG sentences to use the [S] variable. I let TMG change things automatically during the v9 upgrade, but I may have done some myself.  I hope I don't have many in Witness sentences.  But thanks for clearing that up.  It makes sense.

 

Yeah, I saw those of the Refugees list also.  I was happy to see the confirmation that I wasn't too far out in left field.

 

Thanks, Jim, for your work on this transition.