Jump to content


Photo

Conditional Field Source Brackets not Honored?

Source templates

  • Please log in to reply
10 replies to this topic

#1 Paul Harris

Paul Harris

    Advanced Member

  • Members
  • PipPipPip
  • 172 posts

Posted 23 August 2012 - 07:40 PM

I must be missing something. Using the following Source Template:

[Author], <i>[Title]<: [SubTitle]></i> (<[PubPlace]|N.p.>: <[Publisher]|n.p.>, <[PubDate]|n.d.>)<, [Page]>.< [Annotation]>

The Page and Annotation fields are part of the citation detail. If text is entered into the Annotation field it displays properly on screen and in report citations. However, if it is left blank, it is not recognized as a field in reports and prints as literal text: [Annotation].

Yes, I am confused and probably missing something, but I am very familiar with using conditional brackets in templates in RM and other applications. What am I missing? :wacko:

Paul

#2 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6250 posts

Posted 23 August 2012 - 08:15 PM

You will recall from the earlier discussion (about where the Page numbers went) that merging your Free Form source into your custom template did not modify the XML string that contains the Citation Detail, e.g.:
<?xml version="1.0" encoding="UTF-8"?>
<Root><Fields><Field><Name>Page</Name><Value>Belfast ref 1.415 Dec qtr</Value></Field></Fields></Root>
This string does not contain the [Annotation] field name and data and will not until RM has opened Edit Citation on it and saved, even with no value entered in that field. When RM calls the Source Sentence Template, it spits out missing field names as [fieldname], hence you see "[Annotation]" in your footnotes.when the citation has not been opened/edited under new Source Template.

I encountered the same issue with conversion from Free Form to !MyFreeForm3 and developed a SQLite query to globally add the missing field:
-- Add a [ShortPage] definition to every citation of a !MyFreeForm3 source lacking one
-- else you will see the text "[ShortPage]" outputted in RM4 reports
UPDATE CitationTable
SET FIELDS = CAST(REPLACE(CAST(FIELDS AS TEXT),'</Fields>', '<Field><Name>ShortPage</Name><Value/></Field></Fields>') AS BLOB)
WHERE CitationID IN
(SELECT CitationID FROM CitationTable C JOIN SourceTable S USING(SourceID)
	 WHERE TemplateID=YourTemplateID AND CAST(C.FIELDS AS TEXT) NOT LIKE '%<Name>ShortPage%'
)
;
With slight modification (i.e. "Annotation" in place of "ShortPage", the same query could globally fix your "[Annotation]" issue. For more on this see Global Conversion to !MyFreeForm3.

The example from above, so modified with an empty Annotation field, would look like this:
<?xml version="1.0" encoding="UTF-8"?>
<Root><Fields><Field><Name>Page</Name><Value>Belfast ref 1.415 Dec qtr</Value></Field><Field><Name>Annotation</Name><Value/></Field></Fields></Root>

Of course, one might ask why RM reports missing fields in this manner. Is it of any help, especially if there is no easy fix built in? Should it be merely suppressed? Is it a holdover from the debugging stage of Source Template feature development? Maybe a Wish List item would be that RM stop outputting the names of missing fields when it uses the Source Sentence Template in reports, or there be an user option to turn it on and off in Program settings.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#3 Paul Harris

Paul Harris

    Advanced Member

  • Members
  • PipPipPip
  • 172 posts

Posted 23 August 2012 - 09:19 PM

Tom,

Well, all I have to say about this tonight is YUK!

Not a very warm welcome to migraters. I used to wonder why you were so big into back dooring the database. Now I understand you, completely!

Paul

#4 Paul Harris

Paul Harris

    Advanced Member

  • Members
  • PipPipPip
  • 172 posts

Posted 23 August 2012 - 09:32 PM

No, I have to ask one question before retiring, tonight. Tom, if I follow your logic (actually, RM's logic, just your understanding of it) correctly, if I create a custom source template and start using it by creating citations to it, then go back and add a field to the citation detail, then existing citations will have the same printed display problem with that field until those citations are edited? If that is the case, that is lunacy.

Paul

#5 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6250 posts

Posted 24 August 2012 - 06:10 AM

Very logical question, Paul, and not one that I had come to mind before but the answer is affirmative. If there is a field name in the Source Sentence Template that is not present in the Master Source or Citation data (i.e. in the Fields cell of the SourceTable or CitationTable, respectively), RM outputs that field name in square brackets in any report or display that uses the Source Sentence Template. Even in the Source List report where we are accustomed to see the field names of the Source Detail fields so presented in the common Footnote, those absent Master Source fields are also outputted. So, yes, adding fields to a well used custom Source Template has unwanted and unintended consequences.

The other side of the coin is that removing a Source Detail field from the Source Sentence Template and, indeed, even from the Source Fields definitions does not remove the values of that field from the Source List Report (and, possibly, albeit unchecked, neither from the GEDCOM export). So the Source List report will show data that does not come out in report footnotes.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#6 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 8446 posts

Posted 24 August 2012 - 08:40 AM

Confirming I have noted this issue and need enhancement in our tracking system.
Renee
RootsMagic

#7 Paul Harris

Paul Harris

    Advanced Member

  • Members
  • PipPipPip
  • 172 posts

Posted 24 August 2012 - 08:55 AM

Tom, et al.,

I have always been very excited about the power and flexibility of RM's Source Templates. I felt they had given us tremendous flexibility and customization, until now. With that flexibility comes a price, which is unrealized expectations:

I would 'expect' that if I modify a Source Template, ie., rename a field or add a field, that the database would 'automatically' be updated to reflect the change.

I would 'expect' that if I removed a field from a Source Template that I would be asked what to do with the data in use by that field.

I can't even begin to imagine the nightmare scenario that would happen if I were to 'move' an existing field from the Master Source to the Citation Detail!!

Needless to say, I am now very disappointed in my effort to migrate to RM. I guess I had 'unrealistic' expectations. I would like to suggest a Wishlist improvement, but have no idea where to start on this as the problem is so vast and the solution needs to be so comprehensive. :(

Paul

#8 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6250 posts

Posted 24 August 2012 - 08:43 PM

Just did a little test exporting a GEDCOM of a person with a citation post-merge as above in which there is a Master Source field and a Source Details field in the Footnote Sentence Template but the fields are absent from the source and citation data (because the citation has not been subsequently opened by the Citation Editor). Sure enough, the name of the absent Master Source field is outputted in TITL in the square braces. The name of the Source Details field is not outputted in PAGE because all RM does is concatenate all the values found in the Source Details data, separated by semi-colons; the corollary is that even if the Footnote Sentence Template does not use a field that is in the data (e.g., if the target source template in a source merge does not use a field from the secondary source), that value, though unreported in RM footnotes, will be exported to PAGE and will show up when imported to 3rd party software whether wanted or not!

Pretty haphazard...

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#9 Paul Harris

Paul Harris

    Advanced Member

  • Members
  • PipPipPip
  • 172 posts

Posted 24 August 2012 - 10:22 PM

Tom,

Armed with all of this knowledge, where do we go from here? Guess we all need to jump on the SQLite editing bandwagon to truly get our sourcing data cleaned up?

Paul

#10 Paul Harris

Paul Harris

    Advanced Member

  • Members
  • PipPipPip
  • 172 posts

Posted 24 August 2012 - 11:04 PM

Confirming I have noted this issue and need enhancement in our tracking system.


Renee,

Forgive me, but I was suddenly struck with the imagery of the last scene in "Raiders of the Lost Ark," where the camera focuses on the crate containing the Ark of the Covenant as it is being wheeled into the storage warehouse and the camera zooms out to show its vastness and one realizes the 'hopelessness of it ever being seen, again. I am so easily amused. :)

Paul

#11 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6250 posts

Posted 25 August 2012 - 06:57 AM

Armed with all of this knowledge, where do we go from here? Guess we all need to jump on the SQLite editing bandwagon to truly get our sourcing data cleaned up?

Short of RM providing a retro-fix, I see only three ways to correct the unwanted outputs:
  • Open every citation having these unwanted outputs with the Citation Editor and save it to store null values in the field or edit it with desired values.
  • Use a word processor to delete every occurrence of "(*)" (where * reps the wildcard for any string of characters and "()" are actually square braces which I cannot use in this message) in the RTF report. Unfortunately, that would blow away every other use of the square braces and has to be done for every report with citations.
  • Use SQLite as I have shown above to add the absent null fields to either or both the Fields cells of the CitationTable and SourceTable.
1 and 3 fix the problem for a given source/template; 2 is just a bandaid for a given report. 3 is the easiest and fastest, given sufficient knowledge and ability and need not be risky (backup!).

Edited by TomH, 25 August 2012 - 07:00 AM.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.