Jump to content


Photo

Finding sources with blank citations - page number


  • Please log in to reply
18 replies to this topic

#1 TruthSeeker

TruthSeeker

    Member

  • Members
  • PipPip
  • 20 posts

Posted 17 January 2016 - 08:22 AM

How do I find sources with a blank citation? These are free form sources when you hit edit you see a single "page number" line in Source Details and I know I have some blank ones. I can find all sorts of things under search but not page number or citation, I truied "text is blank" but this did not give me the resuts I was looking for.

 



#2 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3872 posts

Posted 17 January 2016 - 09:05 AM

This is a capability that existed in Family Origins and in RM1, RM2, and RM3. But the capability went away starting with RM4 and it's support of source templates. The reason the capability went way is that what we think of as fields in source templates are not actually fields in RM's underlying SQLite database. For that reason, even writing the query you seek in SQLite is a pretty tricky business because to do so you have to be able to parse the way the source template fields are stored. The data is stored as XML, and it's pretty hard to parse XML with an SQL query.

 

I still have lots of citations that I'm cleaning up that use the free form source template, and I frequently find myself trying to search for things in them. The strategy I have the most luck with is to search for something like Source (general)->Footnote contains or Any fact->Source->Footnote contains. But when I do it that way, there is something there that I can look for. It's hard to see how to formulate a search for a blank page number in a free form source. Now that I've said that, Tom will probably whip together an SQLite script that will do the trick for you, and it will probably only take him five minutes or so.  :)

 

Jerry



#3 Laura

Laura

    Advanced Member

  • Members
  • PipPipPip
  • 4276 posts

Posted 17 January 2016 - 10:19 AM

The only way I know to find a blank Source detail, Page number is to use Print on Lists, Source list.

Highlight a Master source on Lists, Source list.

Check on Print.

Choose whether to print all sources or a single source.

Uncheck all the boxes in the left column.

In the right column, check Citation details. Uncheck the other choices.

People with that source linked without an entry in the Source detail, Page number will have just the Name, Record number and fact.

#4 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6424 posts

Posted 17 January 2016 - 11:19 AM

It's hard to see how to formulate a search for a blank page number in a free form source. Now that I've said that, Tom will probably whip together an SQLite script that will do the trick for you, and it will probably only take him five minutes or so.  :)

Did that four years ago  B) . Source List Query
Here's a sample output from SQLiteSpy:
Source_List_Empty_Page.jpg
The example shown is for the !MyFreeForm3 template which also has the Page field. The highlighted cell in the "Cit Fields" column has an empty Page field as confirmed in the expansion of that cell at the bottom where you see the other field in that template. The same script is integrated in RMtrix but it hangs up if there are many citations. To the right, you can see the person and fact type to which the citation is linked and can use that to find it in RootsMagic and edit.
 
Here's a sample output from SQLite Expert Personal running the same script. These are Free Form citations shown.
Source_List_Empty_Page1.jpg

Tom user of RM7630 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#5 TruthSeeker

TruthSeeker

    Member

  • Members
  • PipPip
  • 20 posts

Posted 17 January 2016 - 12:35 PM

Thank you all but rootsmagic is getting more and more confusing to me, there are so many decisions to make. I read reasons not to use shared events, reasons not to use place details and reasons not to use source templates including duplication and lack of gedcom support. Now if I have read these kind answers correctly rootsmagic dropped some support fro free form sources many years ago or did they just forget to put it back in?

 

The more I read the more I don't know what to do for the best, the gereral opinion is to avoid all the whistles and bells and keep to the basic genealogy old fashioned things so I keep within gedcom support but now I am reading that I can't search old style compatible sources from within rootsmagic, duh



#6 Laura

Laura

    Advanced Member

  • Members
  • PipPipPip
  • 4276 posts

Posted 17 January 2016 - 01:42 PM

Just think of all TMG bells and whistles that TMG users used in TMG that didn't import into RM by gedcom that Bruce wrote a conversion program for.

Bruce rewrote RM's gedcom import for FTM users so RM would import FTM's database so users would lose less data.

Did most users of those two programs realize that they could lose data when importing that program's specific features?

At least it is openly discussed on this forum.

RM already has a conversion for Legacy.

Every program has Specific features that do not import into another program by gedcom.

RM users were requesting Shared facts, Source detail like box, and Source templates. And Bruce obliged them.

#7 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3872 posts

Posted 17 January 2016 - 03:46 PM

Now if I have read these kind answers correctly rootsmagic dropped some support fro free form sources many years ago or did they just forget to put it back in?

 

The support for free form sources is still there. The only thing that got lost was the ability to search sources by page number.

 

Jerry



#8 TruthSeeker

TruthSeeker

    Member

  • Members
  • PipPip
  • 20 posts

Posted 17 January 2016 - 06:07 PM

I'm still confused why on the path of progress something would get lost. Laura you seem to be elevating Bruce to some heights but if things are being dropped or forgotten about and Bruce is the driving force behind rootsmagic then I think that is a little misplaced.



#9 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6424 posts

Posted 17 January 2016 - 07:54 PM

This is a capability that existed in Family Origins and in RM1, RM2, and RM3.

 

I don't see the capability to find empty Page fields in FO9 and I skipped from it to RM4 so never noticed the loss.


Tom user of RM7630 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#10 Laura

Laura

    Advanced Member

  • Members
  • PipPipPip
  • 4276 posts

Posted 17 January 2016 - 08:23 PM

I'm still confused why on the path of progress something would get lost. Laura you seem to be elevating Bruce to some heights but if things are being dropped or forgotten about and Bruce is the driving force behind rootsmagic then I think that is a little misplaced.


There is not anything Bruce can do about other programs importing RM specific features. He isn't the other programs' programmers.

It is up to other programs programmers to decide if they are going to accommodate their users importing RM's specific features to fit into their database structure.

Programmers of any program have a choice, keep the program as it has always been or move forward and add specific features their users are asking for that may not be imported into another program.

I do not appreciate your personal comment.

#11 TruthSeeker

TruthSeeker

    Member

  • Members
  • PipPip
  • 20 posts

Posted 17 January 2016 - 08:46 PM

I do not appreciate your personal comment.

I recognize your comment but if something that was gedcom compatible existed in rootsmagic like standard sources and searching for these was dropped in favor of a non standard system then I see this as a negative and not something to defend rootsmagic on.



#12 kbens0n

kbens0n

    Advanced Member

  • Members
  • PipPipPip
  • 3480 posts

Posted 17 January 2016 - 10:06 PM

I recognize your comment but if something that was gedcom compatible existed in rootsmagic like standard sources and searching for these was dropped in favor of a non standard system then I see this as a negative and not something to defend rootsmagic on.

 

GEDCOM compatibility has never changed for Source citation Page numbers; all database contents (blank or filled-in) are conveyed in the exported GEDCOM, as they always were.

What happened was that the configuration of the program's handling of Source records, as it relates to citations (as well as other elements of the program) were rearranged in the graphical interface and database structure, but some limitations in the code structure obviously could not evolve alongside those changes. Thusly, some parts of the records now are not searchable and the amount of programming effort it would take to add additional code as a band-aid and synchronize it with the current user interface likely was not prudent, in light of the already-announced intention to rewrite for universalization across PC's and Mac's. I've said this a number of times before ...if it was just a matter of somewhat relatively straight forward fixes, those two developers would have implemented them. Trust me ;-)


---
--- "GENEALOGY, n. An account of one's descent from an ancestor who did not particularly care to trace his own." - Ambrose Bierce
--- "The trouble ain't what people don't know, it's what they know that ain't so." - Josh Billings
---Ô¿Ô---
K e V i N


#13 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6424 posts

Posted 18 January 2016 - 11:09 AM

GEDCOM compatibility for sources has changed detrimentally else there would not be so many ongoing discussions about loss of structure, corrupted footnotes et al. Reviewing Family Origins 9 on my Windows 10 tablet, it had source fields that corresponded to the standard GEDCOM tags for both import and export. I think RM1-3 did the same. Since RM4, the TITL, AUTH, PUBL tags are mashed together into one value on import and then exported in TITL, thus losing the structure provided by GEDCOM. The RM Basic Book Template corresponds closely to the GEDCOM structure. It is a pity that RM does not provide the option of importing to it and exporting from it tag-transparently. That's an enhancement (restoration?) request I've made in the past.  

 

Searching for a blank Page field in a database structure where Page is a column (as in pre-RM4) is pretty easy even though the developer had not implemented it in FO9. Post-RM3, the Page field is embedded in an XML microdatabase for each citation and the source template system allows the user to define any number of fields, none even named "Page", to be used in a citation's XML microdatabase. Searching by field is therefore a much more challenging proposition, one that the developers have yet to rise to. In RM4, you could not even search sources. Not until RM6 was that ability added with the new Find Everywhere feature. But that is a different kind of search - it searches for matches to the search string in the expanded source footnote sentence, not within the source fields. You cannot search for 'blank' with Find Everywhere. Nor can you even restrict it to search only sources, let alone those using any particular template and any one field within that template.

 

It is possible to search citations for blank Page fields using SQLite and a little knowledge of XML. And it is possible to design a user interface in RootsMagic that would enable users to construct field by field searches of sources. The rest of this post shows what's involved in finding blank Page fields.

 

Here is a snippet of results from the SQLite query for blank page fields:

Waymarks
James B. Acomb-1509
 Birth 1891
 citing 1901 Census of Canada
James B. Acomb-1509
 Residence 1901
 citing 1901 Census of Canada
James Merle Dickeson-1117
 Birth 1911
 citing California Death Index, 1940-1997
James Merle Dickeson-1117
 Birth 1911
 citing Iowa State Census Collection, 1836-1925

The Waymark is the path to follow in RootsMagic to be able to edit the blank Page field, e.g., for the first waymark:

Search

>Person List

>Alt-R

>1509[Enter]

>browse for the Birth event in 1911

>click on the checkmark in the Source column

>double-click on the "California Death Index, 1940-1947" in the Citation Manager

 

The SQLite query that produced the list of citations with blank Page fields was:

SELECT Waymarks 
FROM CitationWay
WHERE Fields LIKE '%>Page</Name><Value/%'
;

A typical citation XML microdatabase with a blank Page field looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<Root><Fields><Field><Name>Page</Name><Value/></Field></Fields></Root>

If Page had a non-empty value, the <Value/> tag would become "<Value>something</Value>".

 

The CitationWay table that was queried was a temporary View created by a much more complex script that essentially adds a waymark to each row in the RootsMagic database CitationTable to make it convenient for the user to get from a row in that table to the RootsMagic Citation Editor that affects that row. In RootsMagic Find Everywhere, the equivalent would be the hyperlink; in this case, the waymark is better than a 'hypolink' - it works if you follow it but it's slow.

 

CitationWay is one of many wayfinding views created by the script "RM6_WaymarksViews.sql" at  

Search - wayfinding from data tables to RootsMagic screens

Tom user of RM7630 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#14 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3872 posts

Posted 18 January 2016 - 01:48 PM

 

In RM4, you could not even search sources. 

 

The search of sources that was introduced with Find Everywhere in RM6 is more explicit that what was available in RM4, but you "kind of" could search sources in RM4 and all subsequent releases by searching for footnote sentences, short footnote sentences, and bibliography sentences. These sentences are not stored in the database. Rather, they are constructed from data that is stored in the database. So what is searchable is the constructed sentences as a whole, not the individual data elements that make up the constructed sentences. Hence, with the Free Form source template you can use the "contains" keyword in the search to search for data from the page number field. That is you are searching the footnote sentence as a whole but the footnote sentence contains the page number field. You might have to be careful in constructing your search not to find a character string that's stored somewhere other than the page number field, but you can usually make it work. What you almost never can do by searching the footnote sentence when using the Free Form source template is to find cases where the page number field is blank.

 

Jerry



#15 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6424 posts

Posted 18 January 2016 - 02:40 PM

Oops, I overstated by saying RM4 could not search sources but, Jerry, in a way you have overstated that it does, even with the "kind of" qualifier. I should have said that RM4 could not effectively search sources. The Search>Person List searches for people, not for sources, whose General or Family (you specify) footnotes contain something. It does not evaluate the footnotes for Alternate Name and Event sources which is a huge hole in coverage. It does not take you to the citation manager, only to the person from which you have to inspect the General or Family citations to find the one that matched.

 

Find Everywhere is the first effective source search from which you can go directly to the Citation Manager from the results. But it has a serious shortcoming because the citation is viewed out of context, i.e., who and what is it evidence for? With the current database design restricting each citation to just one thing, it should be easy to display who and what that is in the Citation Manager heading, much as I did with the Waymarks.


Tom user of RM7630 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#16 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3872 posts

Posted 18 January 2016 - 05:33 PM

Oops, I overstated by saying RM4 could not search sources but, Jerry, in a way you have overstated that it does, even with the "kind of" qualifier. I should have said that RM4 could not effectively search sources. The Search>Person List searches for people, not for sources, ....

 

Yes, I probably overstated just a bit. It is correct that RM searches for people, not sources. But also it is correct that RM searches for people, not places. And it is also correct that RM searches for people, not dates. RM searches for people, not colors. Etc. Which is to say that the searches you can do with RM explorer are all based on people who have certain characteristics (which can include sources, places, dates,colors, etc.). These "certain characteristics" are where my "sort of" came from. But as you indicated, you can't search the source table or the citation table directly except in as much as it can be done with Find Everywhere, you can't search the place table directly except in as much as it can be done with Find Everywhere, etc.

 

I also agree that finding citations with Find Everywhere is of very limited value because the user interface provides you no context.

 

Jerry



#17 TruthSeeker

TruthSeeker

    Member

  • Members
  • PipPip
  • 20 posts

Posted 19 January 2016 - 10:15 AM

OK my readings suggest that the database and the program changed in version 4 so that so that can partly explain the support for this search being left out. I can only hope that it returns in the next version as searching this information is an important consideration.



#18 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3872 posts

Posted 19 January 2016 - 10:56 AM

OK my readings suggest that the database and the program changed in version 4 so that so that can partly explain the support for this search being left out. I can only hope that it returns in the next version as searching this information is an important consideration.

 

RM1, RM2, and RM3 were the "same program with updates". RM4, RM5, RM6, and RM7 are the "same program with updates". RM4 was a complete rewrite, and used a completely different database technology than did previous releases.

 

The updates and improvements in RM4, RM5, RM6, and RM7 are considerable. There are many, many features in RM7 that were not in RM4, but it's still sort of the "same program with updates". The updates in RM1, RM2, and RM3 were not trivial, but they were nowhere near as considerable as the updates in RM4, RM5, RM6, and RM7.

 

The ability to search for a single "page number" in citations was lost in RM4 by the introduction of source templates. Indeed, with many of the source templates, there isn't even a data element called a page number. But whatever data elements are defined in the various source templates, none of them are searchable directly. They are only searchable in the context of searching for complete and mashed together footnote sentences. It would be quite useful, I think, to be able to search based on the data elements defined by source templates. But to so would be a fairly challenging programming task because the number of data elements is unlimited and is user definable. I have no idea if this is something that the RM developers are looking at in the total rewrite that is currently in process. In any case, I think that the only way that the ability to search directly for a single "page number" or its absence would be if the developers added the ability to search for any and all data elements defined by source templates.

 

The issue here is much deeper than just the ability to search for a single "page number" in citations. The introduction of source templates created a similar problem for the import and export of citations via GEDCOM.  As I see it, the problem with GEDCOM essentially is that GEDCOM doesn't have all the tags that are in the source templates or that users can introduce into the source templates. So RM has to take the mashed up footnote sentences that result from the source templates and map them to and from the GEDCOM tags for sources. See Tom's comments above about what happens to the TITL, AUTH, and PUBL tags. The good news is that if you choose to do so, you can achieve clean mappings to and from GEDCOM by using the free form source template instead of the Evidence Explained style source templates, but doing so does not restore the ability to search for a single "page number" in citations. The searching mechanism searches all sources and citations in the same way, which means it is searches the mashed together footnote sentences even for free form source templates.

 

Jerry



#19 Nettie

Nettie

    Advanced Member

  • Members
  • PipPipPip
  • 1688 posts

Posted 19 January 2016 - 12:58 PM

Jerry   a good explanation.  Thanks


Genealogy:
"I work on genealogy only on days that end in "Y"." [Grin!!!]
from www.GenealogyDaily.com.
"Documentation....The hardest part of genealogy"
"Genealogy is like Hide & Seek: They Hide & I Seek!"
" Genealogists: People helping people.....that's what it's all about!"
from http://www.rootsweb....nry/gentags.htm
Using FO and RM since FO2.0