- RootsMagic Forums
- → TreeTraverser's Content
There have been 22 items by TreeTraverser (Search limited from 11-December 18)
I really can't come up with any reason that I would want a non-image file marked as No in that scrapbook tag to be imported into another program.
It's for the same reason you might want to display some photos in your scrapbook, but not others. Suppose you had documents (PDF files) in your scrapbook. Some documents might be private diaries or they may contain sensitive documents like a birth certificate. Other documents might be a family newsletter that you would want to display.
Such a flag is important when I export my genealogy data for use in other applications. I still want to control an item's inclusion in the scrapbook, even if it is not an image. When I export a GEDCOM file, a "scrapbook" flag (_SCBK) is still generated for non-image media items, but it is forced to YES, meaning it is always included.
Although it may not make sense for RootsMagic to display non-image files in its reports, RootsMagic can simply ignore that flag for such media items. The flag is still useful in other applications that can display non-image files in the scrapbook.
I stand by my statement that RM currently exports GEDCOM files with improper source citations. The issue of "unusable" versus "stylistically poor" is not just a matter of taste, but whether one considers the data so disconnected (out of context), that it is not meaningful. You can live with it. I think it's atrocious. To your credit, I am surprised more people don't have a problem with their RM GEDCOM files. I know there are a lot of die-hard, loyal, and exclusive RM fans on this forum and this won't be an issue for them. But, I consider GEDCOM conformance a critical part of any genealogy software and I think there will come a time when it will be an issue for them.
The issue has nothing to do with my software. I keep bringing the issue up because I think it's a problem for anyone that wants to use their RM data with any third-party application. RM is my primary data entry program, but I frequently use Legacy, Family Tree Maker, PAF, etc, to print reports and charts depending on my needs.
Obviously Bruce doesn't see it as a high-priority problem, or maybe even a problem at all. It's been several months and there have been several RM releases. If he would come out and say he's not going to address it, fine, I wasted $20 and countless hours converting my free-form sources to templates. It would be a hassle, but I can move on.
TomH, thanks for duking it out with me. I've never had to justify a problem report so aggressively, especially when the problem seems so obvious. It just goes to show, ones man's trash is another man's treasure.
- the footnote that PAF made up is the concatenation of the common TITL value with the individual PAGE value.
I agree there is nothing to do in this situation but concatenate the TITL with the PAGE to minimize the loss of information. In my program, I also delete all the extra punctuation to make it a little more readable. I'm surprised you think the PAF footnote is acceptable. We'll have to agree to disagree on that point.
- the TITL value exported by RM4 is made up of field values solely from the the Master Source fields
It is now, but only because Bruce deleted the template fields from the TITL that would have been populated by Source Detail fields (e.g., [CivilDivision]). That's the crux of the biscuit. We're talking source templates not sources. Constructing a valid source requires information from the source detail.
I guess another solution would be to restructure all source templates to remove any source detail fields from the title of the source, and keep them as part of the source detail (i.e., part of the footnote).
- the exported TITL value carries at least some of the associated formatting and added words from the Source Template Footnote Sentence Template
- the PAGE value exported by RM4 is made up solely of all the Source Detail field values
Agreed. Currently, it appears to be a catch-all field to avoid complete data loss. In my opinion this is not a proper use for a GEDCOM PAGE tag.
- the exported PAGE value carries no modifiers from the footnote sentence template
Agreed. If it were used for it's intended purpose as a page number, it shouldn't carry those modifiers.
- all of the RM4 source fields are propagated through the export except for Master Source fields that have been omitted from the Source Template Footnote Sentence Template
Ok. How this is done internally is not the issue.
- the style of exported footnotes is necessarily different from native footnotes because of the necessary parsing of Master Source and Source Detail fields from the Footnote Sentence Template into TITL and PAGE lines, respectively,
Agreed. RM is really exporting source templates, not sources. Since templates are proprietary, there's no way to express them in standard GEDCOM. My point is, if the GEDCOM export is RM-specific, include those fields that were removed from the TITL so third-party applications can reconstruct the titles. Ideally that would be in a RM-specific GEDCOM tag like _TITL.
If the GEDCOM is non-RM-specific, Bruce must generate a new source record for every unique combination of source detail fields that would result in a different title. That's the nature of templates. It really seems that Bruce did not fully consider how to export sources when he designed source templates. Or, he left those details to a future release. The last post I saw from him was six months ago, effectively saying there is no problem.
- the style of exported footnotes also differs due to the de-formatting, de-modifying of the Source Detail fields in the export to PAGE.
Agreed, but this is not the root of my complaint. Very simply, I want the title of the source. You may find the source title that PAF displays acceptable. I do not because it's unreadable and confusing. I consider source citations a critical part of genealogical research and these citations reflect poorly on any researcher.
In a true GEDCOM, a TITL tag would contain the complete title of the source. You shouldn't need to refer to the PAGE tag of one or more source citations. I'm curious how PAF displays a bibliography. I'll bet it doesn't append the PAGE tag then. And, if you were to use the data you imported into PAF, how would you select one of these imported sources? How would you distinguish the source that uses 'civil division 1' from 'civil division 2' when you went to add a source to a fact? You cannot. Clearly a different civil division of a census source results in a different source. This is another reason it's unusable.
The source template in my example from earlier this year was using the source template "Census, U.S. Federal (Online images)."
With RM-specific off, the source record in the GEDCOM is:
0 @S4106@ SOUR 1 ABBR Census, 1930, Nottawa, Isabella, MI 1 TITL 1930 US Census, Isabella County, Michigan, population schedule, , ; dig 2 CONC ital images, <i>ProQuest, Heritage Quest Online</i> (http://hpl.lib.al. 2 CONC us/ : accessed ); National Archives and Records Administration micropub 2 CONC lication Series T626, Roll 995.
The source citation (footnote) is:
2 SOUR @S4106@ 3 REFN 000911 3 PAGE Nottawa Township; 37-16; Page 120-B, Line 98; Dwelling 115, Family 116; downloaded; 15 July 2005
You cannot just concatenate the two to create the source. The TITL is missing fields like [CivilDivision] and enumeration district [ED]. Those are the fields Bruce removed, leaving the extra punctuation. The data to substitute into those fields come from the source citation (footnote). Here the [CivilDivision] is Nottawa Township and the [ED] is 37-16. Another source citation (footnote) could reference a different [CivilDivision] and [ED], effectively making the single source record S4106 two different sources.
Sure, the civil division and ED are in the PAGE tag, but how would a third-party application understand that 37-16 is the ED? And, how does one know the date given in the PAGE tag is the date the website was accessed? Do you really find the title "1930 US Census, Isabella County, Michigan, population schedule, , ; digital images, ... Nottawa Township; 37-16; Page 120-B, ..." readable and usable? I think it would confuse other researchers. Besides the atrocious grammar, the data do not make sense because there is no context.
The key here is that RM is exporting a source template not a source. In fact, it is an incomplete remnant of a template. Bruce should provide the missing fields so other software can reconstruct the source, or he should export multiple source records, a unique one for each source citation (footnote) that references a different [CivilDivision], [ED], etc.
What I'm saying is, before you go to the trouble of converting all your sources from free-form to using RM source templates, be aware that as it is now, if you export your data to a GEDCOM file, your source information will be unusable by a third-party application. The information is in the PAGE tag, but third-party applications will not be able to reconstruct the source title in a meaningful (usable) way.
Not every source type is affected. If you can demonstrate your experiment works with the census fact type I used, I'd like to know how you did it.
Remember RM uses source templates and a template contains placeholders used to substitute actual information. When Bruce removed the placeholders from the TITL tag, there is no way to know where to substitute the information from the source citations. The extra punctuation is not from empty fields, but instead from the absence of the placeholders. The source citations do contain the information to be substituted, but you can no longer tell where it goes in the source record's TITL tag. We're not talking about sentence templates like you would construct for fact roles in RM.
Many source citations can refer to the same source record, because the source record is a template. But each source citation can "modify" the source record slightly. That is, the information that is substituted into the placeholders can be different, and this can result in a different title. What this really means is, one source record (template) can represent several different sources, not just one.
So if you remove the placeholders from the source template, RM should generate multiple source records based on the source citations that reference them. Or, provide RM-defined tags that do contain the placeholders so other software can reconstruct the titles. I suspect this issue may be the cause of footnote problems other people are experiencing when they generate their websites.
Your suggestion to convert all my sources to copies of the built-in sources would probably work. That is just a work-around though, besides being a monumental amount of effort for a problem that should be corrected in the first place.
If Bruce doesn't understand the problem, or he doesn't agree that it is a problem (I don't know which), then a word of caution to existing RM users: If you ever plan to export your genealogy via GEDCOM for use in third-party applications, most of your source citations will be unusable. As long as you lock yourself into using RM now and in the future, and you will not share your research with others, there is no problem.
Does anyone know of a way to delete these phantom sources? They are not listed in the source list report.
The ABBR, TITL etc are generated for compatability with software that does not handle the template. If you look through the GEDCOM, there should be FIELD, NAME, VALUE triplets following a _TMPLT in the citation under the INDI or FACT, similar triplets for the Master Source under the 0 @Snn@ SOUR line cited, and the full template specification following 0 _STMPLT with the template ID number (TID) pointed to from the Master Source. You would build your citation from these, not the TITL. RM builds the TITL from these but mishandles null fields by not suppressing the extra punctuation (or else the sentence template itself does not anticipate null values and that can be revised).
Ahh, I did not notice the _STMPLT record when I created the user-defined template. I didn't notice it before because I had all standard templates.
Previously the TITL did contain template fields and it was into those that I substituted the NAME/VALUE pairs. Note that RM does not mishandle null fields. Rather, it does not substitute the NAME/VALUE pairs from the source citation (i.e., those following a _TMPLT in the citation under the INDI or FACT). That's a problem for Bruce because it's not a simple substitution. Other citations will have different values and therefore you cannot have just one source record.
Now I have to work a little harder for my supper. I have to parse the full RM template language (given under the _STMPLT record) to reconstruct the source and bibliography titles. It's not a matter of simply substituting NAME/VALUE pairs into the template fields. It was relatively easy to just substitute the NAME/VALUE pairs when the TITL contained the template fields. That's why I still hope Bruce provides the title, etc., with template fields in user-defined tags under the _TMPLT record.
Or, I can throw away my template processing altogether and hope Bruce exports source records properly, that is, multiple source records rather than just one source template record.
Since the source TITL/AUTH/PUBL/_BIBL tags are not complete, I consider the GEDCOM invalid. Not invalid according to the GEDCOM standard, but invalid in the sense the data are incomplete. A simple solution would be to export the template using user-defined tags, for instance _TITL and _BIBL under _TMPLT in the source record. (Although there is still the problem of missing data in the actual TITL tag etc.)
I added RootsMagic support to my program, GED-GEN, for a forthcoming release. It's in the form of an option that enables RootsMagic-style source templates. I took care to generate unique footnotes, recognize identical footnotes (for ibid processing), eliminate duplicate bibliography entries and create new ones caused by a single source record being used as multiple different sources. Then I noticed all the empty punctuation fields and discovered important template fields had been removed.
I copied a census template to make a user-defined template and I get the same problem:
0 @S5@ SOUR 1 ABBR Test-Name 1 TITL Test-Year, Test-Jurisdiction, test-schedule, , ; test-item, <i>Test-Web 2 CONC site</i> (Test-URL : accessed ); Test-Credit. 1 _SUBQ Test-Year, Test-Jurisdiction, Test-Schedule, , , . 1 _BIBL Test-Country. Test-Jurisdiction. Test-Year, test-schedule. Test-Item. T 2 CONC est-Website. Test-URL : . 1 _TMPLT 2 TID 10000
Or should I export the user-defined template from the Source Templates dialog and create an .rmst file? Again, having an accompanied XML file just to reconstruct the title of a source record is certainly contrary to the GEDCOM standard.
Besides, it's not a matter of changing the GEDCOM export to suit my needs. I've tried to argue the point from several different angles that the GEDCOM file is indeed invalid. It seems however that Bruce and others don't see a problem. But if they do, I'd at least like to know whether they are addressing it.
The template fields were removed from the TITL, AUTH and PUBL tags because those tags are what other programs read when they import sources.
The template fields are still provided in the custom tags RM uses so that RM can read the template information back in.
Yes, the field NAME/VALUE pairs are provided and RM can export and re-import its own GEDCOM. However I suspect that RM ignores the TITL/AUTH/PUBL/_BIBL tags and instead reads RootsMagic.st in order to reconstruct the template. In the future when RM attempts to read an old GEDCOM, and RootsMagic.st has since changed, RM will also be unable to reconcile the older NAME/VALUE pairs with a new (modified) template.
I notice that the "Master Source" template fields are not in the TITL/AUTH/PUBL/_BIBL tags. You did have the "Source Detail" template fields, but have since removed them. If you remove them, you will have to substitute those fields with the actual values from the Source Details too. Not only that, but for every reference where the Source Details are not exactly identical, you must generate a completely new source record. You cannot simply remove the Source Detail template fields from the the TITL/AUTH/PUBL/_BIBL tags and give other software no way to reconstruct the template. I suspect this design is the reason there are problems with RM generating websites with mismatched citations/footnotes/bibliography (http://forums.rootsm...ting-a-website/, http://forums.rootsm...bsite-creation/)
Actually Bruce clearly addressed it in his first response. He expressly stated that he removed them from those standard (not custom) tags to avoid incompatibility with other software.
Removing those template fields is the problem. The following title is now even more incompatible with other software (red fields added for emphasis).
1 TITL Sample Year, Sample Jurisdiction, sample schedule, [missing], [missing]; sample item type
2 CONC , <i>Sample Website</i> (Sample URL : accessed [missing]); Sample Credit Line.
I could read the template ID (TID) field and then parse RootsMagic.st to reconstruct the template, as you suggested May 14. Traditionally GEDCOM files have been self-contained "transmissions." Certainly having to parse an accompanied XML file along with a GEDCOM file would be incompatible with all other software. The two files could also get out of synchronization, meaning the XML could change long after the GEDCOM was generated. So I don't think that is a viable solution.
As in my previous example, I created a sample census citation. Its title in a resulting GEDCOM file is:
1 TITL Sample Year, Sample Jurisdiction, sample schedule, , ; sample item type 2 CONC , <i>Sample Website</i> (Sample URL : accessed ); Sample Credit Line.
Notice the , , ; near the end of the first line. In prior versions there were template fields there, like [CivilDivision]. Those have now been removed, leaving empty punctuation. These template fields were formerly substituted with values from each citation that referenced the source template. Thus each source template could represent numerous different sources. Without the template fields, there is no way to reconstruct these sources. Important information is missing, which makes the source effectively meaningless.
RootsMagician has not acknowledged this is a problem, and there have been a number of RM updates since. So I'm left wondering if the issue is being addressed, or if I'm missing something.
Does anyone have experience in reverting to a previous version of RM? I could use a version that generated the full source citations. I'm concerned that newer RM versions may have made database schema changes that affect my data. In other words, I'd use an older RM version on a database updated for the most recent version.