Jump to content


Photo

No Source Template Fields in GEDCOM


  • Please log in to reply
35 replies to this topic

#21 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6251 posts

Posted 16 June 2010 - 04:24 PM

The ABBR, TITL etc are generated for compatability with software that does not handle the template. If you look through the GEDCOM, there should be FIELD, NAME, VALUE triplets following a _TMPLT in the citation under the INDI or FACT, similar triplets for the Master Source under the 0 @Snn@ SOUR line cited, and the full template specification following 0 _STMPLT with the template ID number (TID) pointed to from the Master Source. You would build your citation from these, not the TITL. RM builds the TITL from these but mishandles null fields by not suppressing the extra punctuation (or else the sentence template itself does not anticipate null values and that can be revised).

For TID<10000, RM does not export the template. All user-defined TIDs start at 10000. It would be a simple change to include any used and then you would have all you need.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#22 Alfred

Alfred

    Advanced Member

  • Members
  • PipPipPip
  • 5734 posts

Posted 16 June 2010 - 04:41 PM

The very last thing in ghe GEDCOM file is:
0 _STMPLT
1 TID 10000

With all of the fields following it.
I suppose if you had multiple user defined sources, they would all be there, BUT, only if you actually cited them. RootsMagic does not bother with exporting things that no one uses.
Alfred

#23 TreeTraverser

TreeTraverser

    Advanced Member

  • Members
  • PipPipPip
  • 48 posts

Posted 16 June 2010 - 05:39 PM

The ABBR, TITL etc are generated for compatability with software that does not handle the template. If you look through the GEDCOM, there should be FIELD, NAME, VALUE triplets following a _TMPLT in the citation under the INDI or FACT, similar triplets for the Master Source under the 0 @Snn@ SOUR line cited, and the full template specification following 0 _STMPLT with the template ID number (TID) pointed to from the Master Source. You would build your citation from these, not the TITL. RM builds the TITL from these but mishandles null fields by not suppressing the extra punctuation (or else the sentence template itself does not anticipate null values and that can be revised).


Ahh, I did not notice the _STMPLT record when I created the user-defined template. I didn't notice it before because I had all standard templates.

Previously the TITL did contain template fields and it was into those that I substituted the NAME/VALUE pairs. Note that RM does not mishandle null fields. Rather, it does not substitute the NAME/VALUE pairs from the source citation (i.e., those following a _TMPLT in the citation under the INDI or FACT). That's a problem for Bruce because it's not a simple substitution. Other citations will have different values and therefore you cannot have just one source record.

Now I have to work a little harder for my supper. I have to parse the full RM template language (given under the _STMPLT record) to reconstruct the source and bibliography titles. It's not a matter of simply substituting NAME/VALUE pairs into the template fields. It was relatively easy to just substitute the NAME/VALUE pairs when the TITL contained the template fields. That's why I still hope Bruce provides the title, etc., with template fields in user-defined tags under the _TMPLT record.

Or, I can throw away my template processing altogether and hope Bruce exports source records properly, that is, multiple source records rather than just one source template record.

#24 TreeTraverser

TreeTraverser

    Advanced Member

  • Members
  • PipPipPip
  • 48 posts

Posted 21 November 2010 - 12:30 PM

I still hope the problem with how RM exports source records via GEDCOM will be corrected soon. As of RM 4.0.9.7, third-party software cannot properly process source information found in RM GEDCOM files.

#25 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6251 posts

Posted 21 November 2010 - 04:37 PM

I still hope the problem with how RM exports source records via GEDCOM will be corrected soon. As of RM 4.0.9.7, third-party software cannot properly process source information found in RM GEDCOM files.


I reviewed the thread to refresh my memory and I think you may be whistling in the wind. There is nothing about the GEDCOM export in this area to be "corrected", as far as I can tell. The true correction removed the names of empty fields from the TITL line which should not have been there in the first place. It's unfortunate that you built a procedure based on a faulty GEDCOM. What you are asking for is an "enhancement" that you can conveniently use to generate an alternative footnote and a bibliography other than what is in the TITL. That's more than RM4 can do when it imports third party GEDCOMs - Footnote, Short Footnote and Bibliography are all populated with the same content, except the value of PAGE is omitted from the Bibliography. Is that not good enough for you?

You got hung up on the extra ", , , " in some TITLs, thinking these were empty fields (like comma-separated variables and that you needed to know what position each field is in. All they represent is a sentence template that has not been designed to hide punctuation when a field in the sentence is empty. The punctuation could be anything, even a static word as in the standard Birth certificate, local level example.

What I have suggested to you in the past that will work without modifying RM4 is to convert all your Sources from built-in Source Templates to copies of same. That results in a RM-specific export that will include all you need to generate custom footnotes and bibliography. Yes, that will be harder for you to work with than if you can get Bruce to provide an enhancement that possibly no more than one person is asking for ;). Has anyone else on this forum said what a boon it would be to them?

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#26 TreeTraverser

TreeTraverser

    Advanced Member

  • Members
  • PipPipPip
  • 48 posts

Posted 21 November 2010 - 06:29 PM

Thanks for your input TomH. I have to agree I'm probably whistling in the wind, but I respectfully disagree with you on your other points. I think I explained the problem adequately in my previous posts. I won't try to explain it all again, except to respond to your points. I'm not asking for a custom enhancement, only that source citations be properly exported via GEDCOM.

Remember RM uses source templates and a template contains placeholders used to substitute actual information. When Bruce removed the placeholders from the TITL tag, there is no way to know where to substitute the information from the source citations. The extra punctuation is not from empty fields, but instead from the absence of the placeholders. The source citations do contain the information to be substituted, but you can no longer tell where it goes in the source record's TITL tag. We're not talking about sentence templates like you would construct for fact roles in RM.

Many source citations can refer to the same source record, because the source record is a template. But each source citation can "modify" the source record slightly. That is, the information that is substituted into the placeholders can be different, and this can result in a different title. What this really means is, one source record (template) can represent several different sources, not just one.

So if you remove the placeholders from the source template, RM should generate multiple source records based on the source citations that reference them. Or, provide RM-defined tags that do contain the placeholders so other software can reconstruct the titles. I suspect this issue may be the cause of footnote problems other people are experiencing when they generate their websites.

Your suggestion to convert all my sources to copies of the built-in sources would probably work. That is just a work-around though, besides being a monumental amount of effort for a problem that should be corrected in the first place.

If Bruce doesn't understand the problem, or he doesn't agree that it is a problem (I don't know which), then a word of caution to existing RM users: If you ever plan to export your genealogy via GEDCOM for use in third-party applications, most of your source citations will be unusable. As long as you lock yourself into using RM now and in the future, and you will not share your research with others, there is no problem.

#27 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6251 posts

Posted 21 November 2010 - 09:47 PM

If Bruce doesn't understand the problem, or he doesn't agree that it is a problem (I don't know which), then a word of caution to existing RM users: If you ever plan to export your genealogy via GEDCOM for use in third-party applications, most of your source citations will be unusable. As long as you lock yourself into using RM now and in the future, and you will not share your research with others, there is no problem.

Why do you say that? Have you tried importing a RM4.0.9.7 GED file to other genealogy software? I just did a no-RM specific export to PAF 5.2.18.0 last copyrighted in 2002. From a quick comparison, I do not see a loss of data from the RM4 citations except for PAF's own truncation of long TITLs. The RM4 Source Template Footnote sentence template controls what is outputted to the equivalent of Free Form which is consistent with GEDCOM, as far as I know, and with most interpretations of GEDCOM by the major softwares.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#28 TreeTraverser

TreeTraverser

    Advanced Member

  • Members
  • PipPipPip
  • 48 posts

Posted 22 November 2010 - 09:05 AM

Even with the "Extra details (RM specific)" option unchecked, the TITL tag is missing information (and contains extra punctuation). The missing information is still placed in the PAGE tag in the source citation, as it is when "Extra details" is checked.

What I'm saying is, before you go to the trouble of converting all your sources from free-form to using RM source templates, be aware that as it is now, if you export your data to a GEDCOM file, your source information will be unusable by a third-party application. The information is in the PAGE tag, but third-party applications will not be able to reconstruct the source title in a meaningful (usable) way.

Not every source type is affected. If you can demonstrate your experiment works with the census fact type I used, I'd like to know how you did it.

#29 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6251 posts

Posted 22 November 2010 - 11:23 AM

Even with the "Extra details (RM specific)" option unchecked, the TITL tag is missing information (and contains extra punctuation). The missing information is still placed in the PAGE tag in the source citation, as it is when "Extra details" is checked

RM4 outputs the Master Source fields as the TITL and the Source Details fields as the PAGE. You get the TITL once with its source ID in the sources area of the GED. Every citation of that source points to the source ID and has a PAGE. What every GEDCOM compatible s/w must do on import is to combine the two for each footnote. How they are stored internally is proprietary. When RM4 imports these non-RM specific GEDCOM tags, it appears to do this:
TITL --> Free Form Master Source Footnote, Short Footnote, Bibliography fields
PAGE --> Free Form Source Details Page Number field
A complete footnote is the concatenation of MS Footnote and SD Page Number.

What I'm saying is, before you go to the trouble of converting all your sources from free-form to using RM source templates, be aware that as it is now, if you export your data to a GEDCOM file, your source information will be unusable by a third-party application. The information is in the PAGE tag, but third-party applications will not be able to reconstruct the source title in a meaningful (usable) way.

How is it unusable? If the citation specific data is in the PAGE and the master source data is in the TITL, is not all the data present in the right places in the GED file for GEDCOM compliant software to be able to construct usable citations? PAF did with my examples, with the exception of truncation of TITL content - that's a PAF limitation, not a GEDCOM violation. Of course, the footnotes from the exported citations are not going to look the same as the native ones because the RM footnote sentence template can re-order and intermingle values from both the Master Source and the Source Details fields. These get separated into TITL and PAGE, respectively, on export.

Not every source type is affected. If you can demonstrate your experiment works with the census fact type I used, I'd like to know how you did it.


Maybe we are converging a little. What census fact type or, to be accurate, census source template are you having a problem with? Why not send me a .rmgc file with one person having citations of this census source?

At the risk of forking the discussion in another direction, maybe there is a more common complaint about the style of the exported TITL and PAGE values over which we may have little or no control. I'm not sure that the Source sentence templates for footnote, short footnote and bibliography have any bearing on what goes into TITL and PAGE. I could imagine the bibliography sentence going into TITL but there is no user viewable, definable sentence template for PAGE and it cannot be extracted from the other three. It would appear that there is an internal PAGE template that might be nothing more than "<SD Field1>; <SD Field2>; ...<SD Fieldn;>.", where "SD" stands for "Source Details" and that it is common to all source templates. An attractive enhancement for those exporting to 3rd party apps would be to have a user-definable PAGE sentence template for each source template.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#30 TreeTraverser

TreeTraverser

    Advanced Member

  • Members
  • PipPipPip
  • 48 posts

Posted 22 November 2010 - 01:46 PM

TomH, I don't think I'm going to convince you there's a problem, but I enjoy trying to explain it from every possible angle. :)

The source template in my example from earlier this year was using the source template "Census, U.S. Federal (Online images)."

With RM-specific off, the source record in the GEDCOM is:

0 @S4106@ SOUR
1 ABBR Census, 1930, Nottawa, Isabella, MI
1 TITL 1930 US Census, Isabella County, Michigan, population schedule, , ; dig
2 CONC ital images, <i>ProQuest, Heritage Quest Online</i> (http://hpl.lib.al.
2 CONC us/ : accessed ); National Archives and Records Administration micropub
2 CONC lication Series T626, Roll 995.

The source citation (footnote) is:

2 SOUR @S4106@
3 REFN 000911
3 PAGE Nottawa Township; 37-16; Page 120-B, Line 98; Dwelling 115, Family 116; downloaded; 15 July 2005

You cannot just concatenate the two to create the source. The TITL is missing fields like [CivilDivision] and enumeration district [ED]. Those are the fields Bruce removed, leaving the extra punctuation. The data to substitute into those fields come from the source citation (footnote). Here the [CivilDivision] is Nottawa Township and the [ED] is 37-16. Another source citation (footnote) could reference a different [CivilDivision] and [ED], effectively making the single source record S4106 two different sources.

Sure, the civil division and ED are in the PAGE tag, but how would a third-party application understand that 37-16 is the ED? And, how does one know the date given in the PAGE tag is the date the website was accessed? Do you really find the title "1930 US Census, Isabella County, Michigan, population schedule, , ; digital images, ... Nottawa Township; 37-16; Page 120-B, ..." readable and usable? I think it would confuse other researchers. Besides the atrocious grammar, the data do not make sense because there is no context.

The key here is that RM is exporting a source template not a source. In fact, it is an incomplete remnant of a template. Bruce should provide the missing fields so other software can reconstruct the source, or he should export multiple source records, a unique one for each source citation (footnote) that references a different [CivilDivision], [ED], etc.

#31 Alfred

Alfred

    Advanced Member

  • Members
  • PipPipPip
  • 5734 posts

Posted 22 November 2010 - 05:03 PM

What I see is that all of the master source fields are concatenated into the TITL and it is probably too long to be imported as a source title.
So, some of the later master source fields will be cut off.

I see nothing wrong with the citation details being listed with the citation as the PAGE.
Any program should be able to put the page field with the master source field for footnotes.

There probably should be another tag below or in place of the TITL tag to contain the entire source text. But, I do not think that the citation details or PAGE should be included there. They are for the items unique to this particular citation.
(Unless you think the entire source text should be repeated every time it is cited.)
Alfred

#32 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6251 posts

Posted 22 November 2010 - 09:09 PM

Okay TT, here's the experiment with the Source Template "Census, U.S. Federal (Online images)". I created a Master Source on this template and filled the fields with "MSn name_appearing_to_the_left" where n is from 1 to 8 numbering the fields top to bottom. I cite this source once and fill the Source Detail fields likewise with "SDn name_appearing_to_the_left" where n is from 1 to 7. I cite it a second time for a son and this time fill in the Source Details fields with just "SD1" to "SD7". These two citations are #1 and #3 in the footnotes from a RM4 Descendant report and cite the second source in the Bibliography. Note that there are two citations of the one source and that the footnotes intermingle the MS fields and the SD fields as per the Footnote sentence template.

I then made a copy of the same template and prefixed the Footnote sentence template with "FN:", the short footnote st with "SFN:" and the bibliography st with "BIB:". I cited this source for the father, filling in the Master Source fields with "MS1" to "MS8" and the Source Detail fields with "2SD1" to "2SD7". This is footnote #2 and is the first source in the Bibliography.

RM4 FOOTNOTES

1 MS2 Year and type, MS3 Jurisdiction, ms4 schedule, SD1 Civil Division, enumeration district (ED) SD2 Enumeration District, SD3 Page ID, SD4 Household ID, SD5 Person of interest; ms5 item type, MS6 Website (MS7 URL : SD6 Access type SD7 Access date); MS8 Credit line.

2 FN:MS2, MS3, ms4, 2SD1, enumeration district (ED) 2SD2, 2SD3, 2SD4, 2SD5; ms5, MS6 (MS7 : 2SD6 2SD7); MS8.

3 MS2 Year and type, MS3 Jurisdiction, MS4 Schedule, SD1, ED SD2, SD3, SD4, SD5.

RM4 BIBLIOGRAPHY

BIB:MS1. MS3. MS2, ms4. MS5. MS6. MS7 : .

MS1 Country. MS3 Jurisdiction. MS2 Year and type, ms4 schedule. MS5 Item type. MS6 Website. MS7 URL : .



Exported with just the Sources box checked and imported to PAF5. Unfortunately, PAF 5 cannot find my PDF printer and there is no copy from its preview screen so I cannot copy and paste the footnotes from its Descedant report. But I can reconstruct what they look like from the Source field and the Citation Detail field that were populated by its import.

PAF 5 Footnotes
1. MS2 Year and type, MS3 Jurisdiction, ms4 schedule, , ; ms5 item type, <i>MS6 Website</i> (MS7 URL : accessed ); MS8 Credi, SD1 Civil Division; SD2 Enumeration District; SD3 Page ID; SD4 Household ID; SD5 Person of

2. FN:MS2, MS3, ms4, , ; ms5, <i>MS6</i> (MS7 : accessed ); MS8., 2SD1; 2SD2; 2SD3; 2SD4; 2SD5; 2SD6; 2SD7


3. MS2 Year and type, MS3 Jurisdiction, ms4 schedule, , ; ms5 item type, <i>MS6 Website</i> (MS7 URL : accessed ); MS8 Credi, SD1; SD2; SD3; SD4; SD5; SD6; SD7






Note that the imported footnotes have all the fields that the RM4 footnote contains - both are missing MS1 which shows up in the Bibliography - but the intermingling has been replaced by MS first, SD second.
Note that the prefix "FN:" I added to the custom template footnote carried through, which suggests that the Footnote sentence template has some bearing on the export. That sentence template looks like this:
FN:[CensusID], [Jurisdiction]<, [Schedule:Lower]>, [CivilDivision]<, enumeration district (ED) [ED]><, [PageID]><, [HouseholdID]>, [Person]; <[ItemType:Lower]|digital images>, <i>[WebSite]</i> ([URL] : <[AccessType]|accessed> [AccessDate])<; [CreditLine]>.
Thus the export has parsed out the MS fields and associated modifiers to one group and the SD fields are merely concatenated with "; " in between. Where do they go in the GEDCOM?
The Master Source fields go here:
0 @S11@ SOUR
1 ABBR MS name - copy US Federal Census (online images)
1 TITL FN:MS2, MS3, ms4, , ; ms5, <i>MS6</i> (MS7 : accessed ); MS8.

The Source Detail fields go here:
1 CENS
2 DATE 1940
2 PLAC Someplace, Somecounty, Somestate
2 SOUR @S11@
3 PAGE 2SD1; 2SD2; 2SD3; 2SD4; 2SD5; 2SD6; 2SD7
The source that was cited twice has one instance of TITL and two instances of PAGE:
1 CENS
2 DATE 1930
2 PLAC Nottawa, Isabella, MI
2 SOUR @S10@
3 PAGE SD1 Civil Division; SD2 Enumeration District; SD3 Page ID; SD4 Household ID; SD5 Person of interest; SD6 Access type; SD7 Access date
...
1 CENS
2 DATE 1930
2 PLAC Nottawa, Isabella, MI
2 SOUR @S10@
3 PAGE SD1; SD2; SD3; SD4; SD5; SD6; SD7
...
0 @S10@ SOUR
1 ABBR Master Source name - Census, US federal (online images)
1 REFN SrcFileName
1 TITL MS2 Year and type, MS3 Jurisdiction, ms4 schedule, , ; ms5 item type, <
2 CONC i>MS6 Website</i> (MS7 URL : accessed ); MS8 Credit line.

As you can see:
  • the footnote that PAF made up is the concatenation of the common TITL value with the individual PAGE value.
  • the TITL value exported by RM4 is made up of field values solely from the the Master Source fields
  • the exported TITL value carries at least some of the associated formatting and added words from the Source Template Footnote Sentence Template
  • the PAGE value exported by RM4 is made up solely of all the Source Detail field values
  • the exported PAGE value carries no modifiers from the footnote sentence template
  • all of the RM4 source fields are propagated through the export except for Master Source fields that have been omitted from the Source Template Footnote Sentence Template
  • the style of exported footnotes is necessarily different from native footnotes because of the necessary parsing of Master Source and Source Detail fields from the Footnote Sentence Template into TITL and PAGE lines, respectively,
  • the style of exported footnotes also differs due to the de-formatting, de-modifying of the Source Detail fields in the export to PAGE.
While there may be some subtle bugs in the parsing of the Footnote Sentence Template to the TITL line, it is the last bullet that I suspect is at the root of your complaint. For example, the loss of the added words in the sentence template "<, enumeration district (ED) [ED]>" that you see in the RM4 footnote but not in the PAGE line. Here, I would agree that there is room for improvement. This looks like unfinished development.

I would question the development strategy if it is the intention to somehow parse the sentence template for the Source Detail fields for mapping to PAGE as has been done for Master Source fields to TITL. That is fraught with problems. I would rather see separate sentence templates for TITL and PAGE for each Source Template.

To illustrate that you currently have some control over the TITL format and content, I revised the Footnote sentence template and got this:
0 @S11@ SOUR
1 ABBR MS name - copy US Federal Census (online images)
1 TITL FN: MS1, MS2, MS3, ms4, ms5, <i>MS6</i> (MS7 ) accessed ; MS8.
Now the missing Master Source field is present, the extra commas are gone. However, the word "accessed" should not be in the TITL but is part of a switch with a Source Detail field (SD6) and should be in PAGE if that field were empty - that's a bug and an example of the challenge of parsing the Footnote template. A workaround would be to delete the word from the template and require that a word be typed into the field. How did the RM4 footnote turn out?

2 FN: MS1, MS2, MS3, ms4, 2SD1, enumeration district (ED) 2SD2, 2SD3, 2SD4, 2SD5; ms5, MS6 (MS7 ) 2SD6 2SD7; MS8.


Compare to #2 at the top of the page.

So TT, are we a little closer to each other in our respective understandings? All the data can be in the TITL and PAGE lines and recovered by other software. What's missing are the Source Detail modifiers and associated words (or the latter may be misplaced). I agree that leaves some room for ambiguity as to what a value means but does not render the citations unusable in third party imports - most data meanings can be inferred from context . To remove ambiguity, there is nothing preventing us from typing in the words "enumeration district (ED)" as part of that field's value and removing them from the template, after modifying to a user-editable Source Template, so that they can show up in both RM footnotes and exported citations. I have even made it possible to do such a type conversion using this SQLite query. Is that still a lot of work? Yes. Should the RootsMagic developers do more on this front? Yes.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#33 TreeTraverser

TreeTraverser

    Advanced Member

  • Members
  • PipPipPip
  • 48 posts

Posted 23 November 2010 - 11:00 AM

Thanks TomH. We are closer in agreement, yet even after your demonstration, I'm surprised you don't draw the same conclusion I do. My May 13 post used a comparable example, but I did not cite a second source for simplicity.

  • the footnote that PAF made up is the concatenation of the common TITL value with the individual PAGE value.


I agree there is nothing to do in this situation but concatenate the TITL with the PAGE to minimize the loss of information. In my program, I also delete all the extra punctuation to make it a little more readable. I'm surprised you think the PAF footnote is acceptable. We'll have to agree to disagree on that point.

  • the TITL value exported by RM4 is made up of field values solely from the the Master Source fields


It is now, but only because Bruce deleted the template fields from the TITL that would have been populated by Source Detail fields (e.g., [CivilDivision]). That's the crux of the biscuit. We're talking source templates not sources. Constructing a valid source requires information from the source detail.

I guess another solution would be to restructure all source templates to remove any source detail fields from the title of the source, and keep them as part of the source detail (i.e., part of the footnote).


  • the exported TITL value carries at least some of the associated formatting and added words from the Source Template Footnote Sentence Template


Agreed.

  • the PAGE value exported by RM4 is made up solely of all the Source Detail field values


Agreed. Currently, it appears to be a catch-all field to avoid complete data loss. In my opinion this is not a proper use for a GEDCOM PAGE tag.

  • the exported PAGE value carries no modifiers from the footnote sentence template


Agreed. If it were used for it's intended purpose as a page number, it shouldn't carry those modifiers.

  • all of the RM4 source fields are propagated through the export except for Master Source fields that have been omitted from the Source Template Footnote Sentence Template


Ok. How this is done internally is not the issue.

  • the style of exported footnotes is necessarily different from native footnotes because of the necessary parsing of Master Source and Source Detail fields from the Footnote Sentence Template into TITL and PAGE lines, respectively,


Agreed. RM is really exporting source templates, not sources. Since templates are proprietary, there's no way to express them in standard GEDCOM. My point is, if the GEDCOM export is RM-specific, include those fields that were removed from the TITL so third-party applications can reconstruct the titles. Ideally that would be in a RM-specific GEDCOM tag like _TITL.

If the GEDCOM is non-RM-specific, Bruce must generate a new source record for every unique combination of source detail fields that would result in a different title. That's the nature of templates. It really seems that Bruce did not fully consider how to export sources when he designed source templates. Or, he left those details to a future release. The last post I saw from him was six months ago, effectively saying there is no problem.

  • the style of exported footnotes also differs due to the de-formatting, de-modifying of the Source Detail fields in the export to PAGE.


Agreed, but this is not the root of my complaint. Very simply, I want the title of the source. You may find the source title that PAF displays acceptable. I do not because it's unreadable and confusing. I consider source citations a critical part of genealogical research and these citations reflect poorly on any researcher.

In a true GEDCOM, a TITL tag would contain the complete title of the source. You shouldn't need to refer to the PAGE tag of one or more source citations. I'm curious how PAF displays a bibliography. I'll bet it doesn't append the PAGE tag then. And, if you were to use the data you imported into PAF, how would you select one of these imported sources? How would you distinguish the source that uses 'civil division 1' from 'civil division 2' when you went to add a source to a fact? You cannot. Clearly a different civil division of a census source results in a different source. This is another reason it's unusable.

#34 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6251 posts

Posted 23 November 2010 - 03:30 PM

TT, we really have to be accurate, careful and consistent in our terminology or this whole dialogue is at cross-purposes. I'm on a short leash at the library so this will be short.

Your original complaint was that you had built an enhancement of your software to take advantage of fields that onec upon a time existed in RM4 exports in the TITL and/or PAGE lines and these suddenly disappeared, rendering that feature unusable. You requested that they be restored and were advised that was impossible because they screwed up the GED for all other GEDCOM compatible software and should not have been there in the first place. You then went on to say that the GED produces citations now unusable by any third party software if people use Source Templates. That is the point I took issue with and demonstrated that third party s/w does produce usable citations from the export, albeit stylistically poor. You go on to say what RM should be doing to export sources and citations that are stylistically consistent with the way they look in RM. That's the fork in the road I was talking about earlier. There are two issues to be addressed:
  • Stylistically poor export of sources and citations in standard GEDCOM format using the TITL and PAGE lines. This is the common or popular problem affecting the many.
  • Failure to export within the GEDCOM sufficient data for a third party s/w to reconstruct sources and citations consistent with the style defined by RM4 Source Templates. This is your issue.

The two are independent of each other and must not be confused.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#35 TreeTraverser

TreeTraverser

    Advanced Member

  • Members
  • PipPipPip
  • 48 posts

Posted 23 November 2010 - 04:58 PM

TomH, I think we are at an impasse. I do think it is constructive to explore an issue by arguing different views, up to a point. As I look over my posts from the preceding months, I see I am saying the same things over and over.

I stand by my statement that RM currently exports GEDCOM files with improper source citations. The issue of "unusable" versus "stylistically poor" is not just a matter of taste, but whether one considers the data so disconnected (out of context), that it is not meaningful. You can live with it. I think it's atrocious. To your credit, I am surprised more people don't have a problem with their RM GEDCOM files. I know there are a lot of die-hard, loyal, and exclusive RM fans on this forum and this won't be an issue for them. But, I consider GEDCOM conformance a critical part of any genealogy software and I think there will come a time when it will be an issue for them.

The issue has nothing to do with my software. I keep bringing the issue up because I think it's a problem for anyone that wants to use their RM data with any third-party application. RM is my primary data entry program, but I frequently use Legacy, Family Tree Maker, PAF, etc, to print reports and charts depending on my needs.

Obviously Bruce doesn't see it as a high-priority problem, or maybe even a problem at all. It's been several months and there have been several RM releases. If he would come out and say he's not going to address it, fine, I wasted $20 and countless hours converting my free-form sources to templates. It would be a hassle, but I can move on.

TomH, thanks for duking it out with me. I've never had to justify a problem report so aggressively, especially when the problem seems so obvious. It just goes to show, ones man's trash is another man's treasure.

#36 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6251 posts

Posted 24 November 2010 - 01:47 PM

TomH, I think we are at an impasse. I do think it is constructive to explore an issue by arguing different views, up to a point. As I look over my posts from the preceding months, I see I am saying the same things over and over.

Not an impasse - a fork in the road. There are two issues. Yes, you are saying much the same thing, but have conflated the two issues. What's right for one is irrelevant to the other.

I stand by my statement that RM currently exports GEDCOM files with improper source citations. The issue of "unusable" versus "stylistically poor" is not just a matter of taste, but whether one considers the data so disconnected (out of context), that it is not meaningful. You can live with it. I think it's atrocious. To your credit, I am surprised more people don't have a problem with their RM GEDCOM files. I know there are a lot of die-hard, loyal, and exclusive RM fans on this forum and this won't be an issue for them. But, I consider GEDCOM conformance a critical part of any genealogy software and I think there will come a time when it will be an issue for them.

This is Issue 1 and on these points we have some congruence if I interpret your black & white imagery in shades of grey. Some source templates (I don't know how many) result in better or worse looking exported footnotes. But what is exported does conform to GEDCOM - GEDCOM says nothing about how to write a source description and a citation of it other than that the first (RM's Master Source data) goes into TITL and the citation specific details (RM's Source Details data) goes into PAGE.

The issue has nothing to do with my software.


But what you have been saying over and over again, i.e., "give me back field names in TITL or under some proprietary GEDCOM tag containing the template so I can reconstruct the source in GED-GEN" has everything to do with your software and nothing to do with other GEDCOM compatible genealogy software. This is Issue 2 and has nothing to do with Issue 1. How did you start this thread? ..with the subject "No Source Template Fields in GEDCOM". Issue 2 is not a bug but an enhancement request.

I keep bringing the issue up because I think it's a problem for anyone that wants to use their RM data with any third-party application. RM is my primary data entry program, but I frequently use Legacy, Family Tree Maker, PAF, etc, to print reports and charts depending on my needs.

This is Issue 1 again and we agree that there is a problem with the exported citations. I have provided the detailed description of what appears to be going on with the GEDCOM standard TITL and PAGE export of sources and citations that use templates. I intend to start a separate thread on this issue so that it is no longer conflated with Issue 2 in this thread. I will also submit a problem ticket.

Obviously Bruce doesn't see it as a high-priority problem, or maybe even a problem at all. It's been several months and there have been several RM releases. If he would come out and say he's not going to address it, fine, I wasted $20 and countless hours converting my free-form sources to templates. It would be a hassle, but I can move on.

I have no idea what Bruce is doing or thinking. It's been several months since you brought up Issue 2 (sorry, I ordered them according to my perception of importance, not chronology) and it is fundamentally a request for an enhancement. I can see that taking lower priority than bug fixing. It's only in the November exchange between you and me that Issue 1 has been clearly described - it definitely is due to one or more bugs, incompleted development, and, maybe underlying that, a bad strategy. I have said in another thread months ago that I wished for a source citation converter from templated source to free form so that I could use the template as an input form cum checklist but then have the freedom to fine tune the wording, order, punctuation in Free Form. I'm just not sure that the closed template system is a good strategy and now use only editable copies. But that's another issue...

TomH, thanks for duking it out with me. I've never had to justify a problem report so aggressively, especially when the problem seems so obvious. It just goes to show, ones man's trash is another man's treasure.

You're welcome. Only trying to bring clarity and understanding to a confusing case and to learn from the process. I think your burden is that you started from a faulty RM GEDCOM, then lost a 'feature' you relied on that was actually a bug, misconstrued the intent of TITL and PAGE as a result, failed to distinguish (at least in your messages) the poor looking exported source citations from your original complaint (what I see as two independent issues) and continue to describe as "the problem". I hope that separating the two issues into separate threads will be clearer to all.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.