Jump to content


Photo

Adventures in Extreme Splitting


  • Please log in to reply
66 replies to this topic

#1 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3534 posts

Posted 07 April 2013 - 01:05 PM

As has been discussed in these forums many times before, most researchers are either lumpers or splitters when it comes to source management. I have been an extreme lumper, and I seem to be moving towards becoming an extreme splitter. I've mentioned this change several times before on this forum, but I've never put the why and how all in one place before. This will be an attempt to do so.

One problem in trying to put a message like this together is that so many different people use such different language from each other when talking about things like sources and citation. Just what is a source? Just what is a citation? Just what is a source detail? Etc. And RM itself seems to be very inconsistent in this regard within its various options and menus and features. I shall start with the following conventions, and I may add to them as we go along.
  • A source is usually a "physical" document such as a book or a marriage record. I put the word "physical" in quotes because these days a book might be an e-book and a marriage record might be a file on a computer. And a marriage record might be an image of an original marriage record, it might be a Word document where somebody has transcribed an original marriage record, or it might be an image of the Word document after the Word document has been printed out. (In making note of the different ways in which a marriage record might appear as a source, we also make note of the fact that there are all kinds of ways we might talk about the overall quality of sources. For example, a document may be primary or secondary, a document may be original or derivative, we need to consider the provenance of documents, etc. These are all important issues, but the quality of a source is not the same question as what is a source. Also, sometimes a source might be an interview with a person, and somehow or other you have to make the interview into a document yourself.)
  • A citation is a footnote or an endnote that appears in some kind of report.
Given these conventions for source and citations, a citation for a book might appear as something like the following.

John Doe. The Doe Family of Boondock County (John Doe, 1999), page 27.

RM itself gets into all kinds of other terminology such as Master Text, Source Text, Source Comments, Source Details, Detail Text, Research Notes, Comments, Quality, and Repository. In putting this message together, it will be necessary to get into some of those items at least a little bit, but I will be avoiding them to the maximum extent possible because the terminology is so confusing.

One more thing to point out before getting into the minutia of source lumping and splitting is that a lumper and a splitter might both produce the same report from RM and the citations in the two reports might be indistinguishable. Which is to say lumping and splitting have to do with the management of source information within a genealogy program and need not have any effect at all on how citations appear in reports.

A published book such as my mythical Doe family book is both an excellent example and a poor example to use to start talking about lumping and splitting. It's a good example because it's really simple, and it's a bad example because source lumpers and splitters would probably treat the book the same in RM. The basic RM concept is that whatever you as a user think of as a single source in the real world becomes a single Master Source in RM. So the Master Source would be "John Doe. The Doe Family of Boondock County (John Doe, 1999)" without the quotes. And the "page 27" without the quotes part of the citation would become what RM calls Source Details. So far, so good.

Another way to think about the same thing (and one of the Webinars about sources makes this very clear) is to think of the Master Source as the part of the citation that is reused and to think of the Source Details as the part of the citation that is not reused. Well, there is sort of a glitch at this stage of my message because in fact the Source Details may also be reused in a certain sense. Which is to say, there may be several different individuals or several different facts for which you enter data into your RM database based on what you find on page 27 of the John Doe's book. RM offers a way in which Source Details can be reused in the sense that a citation can be memorized and having been memorized can be pasted many times.

Herein lies my dilemma. A Master Source in RM can be changed in the Source List and the change automatically applies to the "reusable" part of every citation that uses that Master Source. But there is no equivalent capability to change all the occurrences where Source Details have been memorized and pasted. I have spent so many countless hours fighting with this problem, that I finally decided that if I would move all source information into RM's Master Source area and if I would place no source data whatsoever into RM's Source Details area that the problem would be solved. So that's the direction I'm moving.

Before moving on to an example such as a marriage record that is not a book, I should point out that even with the simple example of a book there might be some lumpers who would differ from some splitters about what is a source. Suppose for example, that our mythical author John Doe was the son of Thomas Doe and Sarah Williams, and that he wrote a second book for which a bibliographic entry might be something like the following.

John Doe. The Williams Family of Boondock County (John Doe, 2005).

Under these circumstances, a lumper might treat John Doe as the source. And if so, then John Doe would become the Master Source in RM and the information about the book, whether the Doe family book or the Williams family book, would be entered into the Source Details area of RM. This is the approach that I have tended to take in order to keep my Source List in RM as short as possible. But again, I finally have reached the point where the problem of changing every occurrence of a citation in order to fix data in the Source Details area is so severe that I'm willing to become an extreme splitter and move all source data into the Master Source area of RM.

For marriage records, being an extreme splitter means that every marriage record is its own separate source which becomes its own separate RM Master Source, even if I have several different marriage records from the same courthouse or from the same marriage book or from the same roll of microfilm. So be it. However, this concept does not mean that if one "marriage record" originated on two or three pieces of paper that I have to treat each of these separate pieces of paper as separate sources.

In modern times, I usually find three separate pieces of paper for one "marriage record". The first piece of paper is the application for license. The second piece of paper is the license itself - really, it's a license authorizing a minister or justice of the peace to perform the ceremony rather than a license authorizing the couple to be married. The third piece of paper is the document that's returned to the courthouse to confirm that the marriage took place. Sometimes the second and third document are actually the front and back of the same piece of paper. But in any case, the important thing to make note of here is that even with extreme splitting it does not have to be the case that each source is a single piece of paper.

This brings us to naming the Master Sources so that they are reasonable to manage. RM does not offer us any structure at all in this regard. The list of Master Sources is a single list that's very long and flat and that's sorted in alphabetic order. The only thing we can do is name Master Sources in such a way that we can find them easily. These Master Source names having nothing to do the way citations appear in reports. They are only visible to us as RM users working within our databases. The convention I came up with is "*marriage LN1, GN1 and GN2 LN2" without the quotes, where LN is last name and GN is given name. I'm using an * as the first letter of all my "new style" Master Sources to cause them to sort in front of all my "old style" Master Sources. The first spouse listed is the one I'm related to and the second spouse listed is the one I'm not related to. If I'm related to both or neither, I list the groom first.

Nothing about the state or county or year or anything like that is in the name of the Master Source, nor does it need to be. That kind of information can and does appear as appropriate in the footnote area of the Master Source.

Finally, that brings us to the storage and naming of the image files associated with my marriage sources. Most typically I have stored my image files in nested subfolders either by family grouping or by physical location as appropriate. But with my new way of managing sources and their associated Master Sources, it seems appropriate to store and name image files the same way. So I have one big folder for storing image files associated with marriages called Marriages. Within that one big folder I name the image files the same way as I name the Master Sources except without the *. In the typical marriage record where there is more than one original piece of paper, I combine all such images into a single file by creating a multiple image TIF file. The free program Irfanview does a very nice job of doing so. So I have one TIF file for each Master Source for each marriage record, each TIF file containing all the relevant images for that marriage.

Generally speaking, I do all the data entry and other prep work for such a marriage source in the List Sources part of RM rather than in the Edit Person part of RM. I include a complete transcription of the marriage record in the field of Master Text called Source Text, and I link to the image file for the marriage record from the Master Source. Then, I use the Master Source as a citation as required. I still have to Memorize and Paste the citation to get it all the places it needs to go. But I having done so, any subsequent corrections or additions only have to be made once, namely to the Master Source for the marriage record.

Finally, I should mention that one rather obscure feature of RM is extremely useful in this kind of extreme source splitting. Namely, a Master Source can be duplicated. So if I get any one Master Source "just right", I can duplicate it from within List Sources and use the copy sort of as a "template" to assist in making a new Master Source. It's not a real template in the sense of the real source templates, but it still makes data entry and standardization much easier.

Jerry

#2 John_of_Ross_County

John_of_Ross_County

    Advanced Member

  • Members
  • PipPipPip
  • 661 posts

Posted 07 April 2013 - 04:45 PM

I would also note that marriage licenses in Ohio can be misleading. The license for my parents was issued in Ross County, Ohio. They drove to Ironton, Lawrence County, Ohio where my great uncle was a Methodist minister and were married in the parsonage. The license indicates that my great uncle lived in Ironton, but there is no specific statement that the marriage took place in Ironton. I think I have read that a license issued in any county in Ohio is good statewide.

On the other hand, the newspaper announcement for my grandparent's wedding in Fairfield County, Ohio, indicates that the minister was from Marion County, Ohio, about 85 miles to the northwest. Beware of jumping to conclusions.

#3 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6218 posts

Posted 07 April 2013 - 09:55 PM

Jerry, this is a fascinating explanation of the problems you have experienced, the analysis thereof and the conclusions you have reached. I suspect/predict these additional outcomes from extreme splitting, i.e., no fields in Source Details:
  • GEDCOM exports of templated sources should be much closer to a perfect replica of the Footnote sentence as it appears in RM reports and not mangled as they are now.
  • Reduced opportunity for exploiting the Short Footnote sentence in RM reports. (Maybe not, on 2nd thought; if all it does is look for matching footnote strings preceding, it would not matter whether a field was in Source Details or Master Source.)
  • RM reports Bibliography section becomes useless or, at least, woefully verbose. (Likewise)
On the first point, I'm curious to check the result after doing some investigation into the complexity of the built-in source templates. Complexity may be indicated by the number of fields in total and the number of those that are in Source Details; I would expect templates that are high in both to be more complex but there are other factors that contribute to the mangling on export. One is having some Source Details fields preceding Master Source fields in the Footnote sentence template. Another is having words in the Footnote sentence template that are triggered with a non-empty Source Detail field as these are dropped on export. So far, I have done the more simple field count and the following templates stand out at the extremes:

Most fields = 17
Census, Canada (online images)
Atlas, online images
Book, Image copy (online, reprint)
GEDCOM File (downloaded)
Most Source Details fields = 11
Family Trees, Documented
Interview, privately held
Least Source Details fields = 0 (i.e., already for extreme splitting, in descending number of total fields)
Artifact, Family, photographed (privately held)
Family Bible Record
Bible Records
Letter, Historic, private
Photo, Portrait, Private (Annotated with Provenance)
Genetic Test (DNA) report
Legal Document, unrecorded (family copy)
Photo, Portrait, Private, scanned
Artifact, privately held
Broadside, original
School Records (student transcripts)
Artifact, archived
Photo, Portrait, Archived (Annotated)
Maps, Topographic
Broadside, reprint
Maps, Historic
Slip Laws, state
Military, Muster Rolls (manuscript)
Slip Laws, Federal

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#4 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6218 posts

Posted 08 April 2013 - 12:29 PM

I have posted a SQLite query that provides the field counter that I used to characterize the source templates to Source Templates - Field Count.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#5 RWells1938

RWells1938

    Advanced Member

  • Members
  • PipPipPip
  • 215 posts

Posted 08 April 2013 - 06:03 PM

I am adding my 2 cents worth? Since I could not find a definition that described a source and have anyone else agree I did my own.

I have only 1 master source template and all of my Master sources use this same template.


Source: A supplier of information which answers the question "How Do I Know That.

Source Description: Enough information to be able to find the object. (RM6 Master Source)
4 Fields - Descriptive Title, Originator, Publisher, Reference Number.

Source Detail: Enough information to be able to find the exact information within the source. (RM6 Source Detail)
This field I use a external script to format.

Source Text: The transcription of the information found. (RM6 Source Research Notes)
This is the actual text of what I have found.

Source Comment: Comment about the source by the researcher. (RM6 Research Notes Comments)
This is my comments about the source.

Source Repository: Is the physical location where the source was viewed. (RM6 Repository)

I have 2 other fields that I have added Digital Copy and Paper Copy these contains the id I have given to the copy if I have one. I have also started using the RM Web tags but not sure if they are worth it yet.

I know this will not fit others but for me it works. It is simple and the output (gedcom and reports) gives the information location of what I found and where and in most cases what I found. I found that ESM books and other methods were to complicated for me so I was spending more time sourcing than research and most who read them did not know what was going on.

Here is a sample:

Master Source and Source Details:
1850 U.S. Federal Census Population Schedule, Kentucky, Bath County, Originator: U.S Bureau of the Census, Publisher: Ancestery.com, Orem, Utah, Division 2, Page Number 33, Lines 28-39, Household Number 229, Household of James Wells, (recorded 28 February 2012).

Research Notes:
James Wells, age 49, M, W, farmer, POB Kentucky
Rebecca Wells, age 27, F; W, POB Kentucky
Joseph Wells, age 21, M, W, labor, POB Kentucky
William Wells, age 19, M; W, labor, POB Kentucky
James Wells, age 16, M, W, labor, POB Kentucky
Alexander Wells, age 14, M, W, POB Kentucky
Elizabeth Wells, age 10, F, W, POB Kentucky
Mary Wells, age 7, F, W, POB Kentucky
Thomas Wells, age 5, M, W, POB Kentucky
Robert Wells, age 4, M, W, POB Kentucky
Benjamin Wells, age 2, M, W, POB Kentucky
Joseph Jones, age 23, M, labor, POB Kentucky

Comments:
Joseph Wells age may be 24.
Living next door is Samuel McWorthy

Repository:
Ancestory.Com
360 W 4800 N
Provo, Utah 84604
United States

Enjoy
Roger

#6 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3534 posts

Posted 08 April 2013 - 07:21 PM

Reduced opportunity for exploiting the Short Footnote sentence in RM reports. (Maybe not, on 2nd thought; if all it does is look for matching footnote strings preceding, it would not matter whether a field was in Source Details or Master Source.)


I always use endnotes rather than footnotes. My experience with endnotes is that multiple identical citations combine perfectly into the Long Footnote sentence (which in effect becomes a Long Endnote sentence). Therefore, Short Footnote sentences do not even come into play as Short Endnote sentences. Extreme splitting as described in my initial message in this thread does not change this behavior, so all is well with respect to the way I list citations in my reports. Under these circumstances, whether the the Short Footnote sentence is properly exploited or not and even the very content of the Short Footnote sentence is completely irrelevant.

There are several reasons why I use endnotes rather than footnotes, and I haven't even looked at footnotes in quite a while. The biggest reason for not using footnotes is that there is less opportunity for multiple duplicate citations to combine because they might be on different pages of a report. But because of this issue and because I haven't looked at true "bottom of the page" footnotes in quite a while, perhaps I never really looked deeply enough into how true "bottom of each page" footnotes are supposed to work. It seems that they combine not at all. So if one particular report page has three identical citations, there will be three separate footnotes at the bottom of the page - one long footnote sentence and two short footnote sentences that are identical. The behavior is the same directly in RM and also in an RTF file with Microsoft Word. The behavior is also the same with my extremely split sources or with more typical and less extremely split sources.

Jerry

#7 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3534 posts

Posted 08 April 2013 - 07:42 PM

RM reports Bibliography section becomes useless or, at least, woefully verbose. (Likewise)


True, and I don't really have a solution except always to create a list of endnotes rather than a Bibliography. Indeed, with extreme splitting it will always be the case that a Bibliography will have exactly the same number of entries as will a list of endnotes.

On the other hand, as an extreme lumper I could not produce a useful Bibliography, either. For example, one of my lumped Master Sources is "Death Certificate" without the quotes. With such a lumped Master Source, all the other citation data has to appear as Source Details and the string "Death Certificate" does not make for a very useful Bibliographic entry. :)

It's interesting to consider that the production of a proper Bibliography seems to require a balanced approach between extreme lumping and extreme splitting. I have sometimes found it both enlightening and distressing to produce a Bibliography from my database. The results are fairly embarrasing, even though I'm very satisfied with the list of Endnotes produced from my database.

Jerry

#8 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6218 posts

Posted 08 April 2013 - 08:15 PM

My experience with endnotes is that multiple identical citations combine perfectly into the Long Footnote sentence (which in effect becomes a Long Endnote sentence). Therefore, Short Footnote sentences do not even come into play as Short Endnote sentences.

Indeed, with extreme splitting it will always be the case that a Bibliography will have exactly the same number of entries as will a list of endnotes.

Two sides of the same coin or the complementarity of the Short Footnote sentence and the "Reuse endnote numbers where possible" option for Endnotes. Short Footnotes are printed in Endnotes if this option is deselected and the endnotes will be generally greater in number than the Bibliography. The same number of endnotes in long and short form will be printed as would be the number of footnotes. Presumably the same algorithm is used to detect that a citation is a repeat and, depending on the checkbox, output either the Short Foot/Endnote sentence with a new number or reuse the superscript number of the initial Full Endnote sentence.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#9 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6218 posts

Posted 08 April 2013 - 09:25 PM

I suspect/predict these additional outcomes from extreme splitting, i.e., no fields in Source Details:

  • GEDCOM exports of templated sources should be much closer to a perfect replica of the Footnote sentence as it appears in RM reports and not mangled as they are now.

Confirming that, when all fields in the full Footnote sentence template are from Master Source and none from Source Details, RM6 (and likely 4 and 5) export exactly the same full Footnote sentence to the standard GEDCOM TITL tag as is displayed in the Edit Source screen. Nothing is exported to the PAGE tag. So 'ultimate' splitting is advantageous over many of the built-in source templates for export as the latter get mangled.

That said, my test was with the Census, US Fed - Online images template and an edited copy of it, as I used in Source Templates, A Comparative Example.pdf. In practice, I cannot imagine splitting to the level of the Person of interest | Line number, nor even to the page number. Jerry, how are you treating census sources?

No doubt there has to be a tradeoff between lumping and splitting that varies with the type of source, as we have seen even in the built-in templates where some have no Source Detail fields (ultimate split) while some have many (although there is no 'ultimate lump', i.e., no Master Source fields). Which leaves us with the unhappy situation of:
  • Sources having Source Detail fields get corrupted on export
  • Export/Import does not even utilize/preserve the full set of GEDCOM tags, ignoring AUTH, PUBL
  • Sources get corrupted on html pages
  • Citation Manager Memorize and Paste creates independent citations with no means to subsequently edit all instances at the Source Detail level
These are issues that, were they resolved, would likely bring you back from the extremes to a practical middle ground.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#10 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3534 posts

Posted 09 April 2013 - 09:02 AM

In practice, I cannot imagine splitting to the level of the Person of interest | Line number, nor even to the page number. Jerry, how are you treating census sources?


Let me answer somewhat obliquely by first describing the sourcing situations that I have already switched over to extreme splitting.
  • Marriage records as previously described.
  • Newspaper announcements. So far, I only have one such example. As I get more of them, I may think of something better to call them. This particular one is is from the Harrisonburg Rockingham Register, 23 Apr 1868, p. 2 and that's what's in the name of the Master Source. I have the image of the article linked to the Master Source as described for marriage records. I have transcribed the article into the Source Text field of the Master Source as described for marriage records. To give you some sense of what this is all about, here is the full text of the newspaper announcement: "MRS. MARY ANN WHEELER :-- Take Notice that on the 9th day of May, 1868, at the Clerk's Office of Rockingham county, Virginia; I shall take the deposition of A. Wilhite and others, to be read as evidence for me in a chancery suit now pending in the Circuit Court of said county, wherein I am complainant and you are defendant. April 9th Peter W. WHEELER." Peter Wheeler was suing his wife Mary Ann Bryan for divorce because she was consorting with another man. The deposition was in support of his suit. Divorces were rare in those days, and it's at least a little curious to the modern eye that such newspaper announcements were a part of the process. I suppose I could mention Peter Wheeler or his wife (or both) in the name of the Master Source, but listing it the way I have done makes sense to me for now. As described for Marriage records, I have created a single flat (no nested subfolders) Windows folders to hold all the images for all newspaper announcments, and the image file name will mirror the Master Source name.
  • Obituaries. Here is an example of a Master Source name for an extremely split obituary source: "*obituary, Cox, Mary Elizabeth Rader, The Greeneville Sun, November 02, 2011." without the quotes. So within obituaries, the primary sort is lastname, firstname. I have a single flat folder called Obituaries for all obituaries, although it's common now not to have images because the obituaries are so often online. The Source Text field includes a transcription of the obituary text. The process follows the same model I developed for marriage records.
  • A person. I'm still thinking about this one, but to me the source (the "physical" source, that is) is the person no matter what the media is - a personal e-mail from them, an old fashioned personal letter from them (yes, I have a few of those), a personal interview with them (either face to face or via telephone). Here are a couple of examples of Master Source names, slightly sanitized for privacy: "*person Doe, Jane, e-mail 3 Oct 2006" without the quotes, and "*person Doe, John, rootsweb.com postem 2007-07-09 22:21:57" without the quotes. I'm still working on getting associated Long Footnote sentences to be in full compliance with Evidence Explained or other authorities, but the nice thing about it is that any adjustments I need to make in this area only have to be made in the Master Source area of RM and nowhere else in RM. I have a single flat folder named Persons to hold any media files. For example, with e-mails I can save an EML file which will retain all the formatting and attachments of the original e-mail. And of course I copy and paste the text of the e-mail into the Source Text field of the Master Source. And equally obviously, because of the extreme splitting I often have several different Master Sources associated with the same person - multiple e-mails from the same person, for example.
  • Newspaper wedding announcment (not to be confused with a courthouse marriage record). I'm not quite sure yet how this is different than just a plain old newspaper announcement, and I may combine the two categories together. But if I change my mind, all I have to do is to rename the Master Source. I will not have to edit a single person anywhere in my database. Here's a real example of a Master Source name, sanitized for privacy: "*wedding announcement - Smith, Jane and John Doe, Daily Press, Serving Hampton Roads, Virginia, 6 Jul 2008." without the quotes. I transcribe the newspaper announcement into the Source Text field of the Master Source and connect any media files to the Master Source the same way as for any of my categories of extremely split sources.
Which brings us to Tom's question about census sources. The short answer is that I haven't decided yet how to incorporate census sources into this new scheme. I'm certain that I will not split down to the individual census line. I will probably split instead down to the individual census page. That's fine for organizing the images and connecting them to Master Sources, but it begs the question of transcriptions. I have no intention of transcribing whole census pages when in fact I'm only interested in one family from that page. On the other hand, it is common that a given census image of interest will have more than one family of interest. So I will probably transcribe into the the Source Text area of the Master Source all the families of interest from that page. And for sure, I will transcribe the heading information (state, town, district number, date, page number, etc.) for each census page into the Source Text area of the Master Source for that census page.

Jerry

#11 zhangrau

zhangrau

    Advanced Member

  • Members
  • PipPipPip
  • 1510 posts

Posted 09 April 2013 - 10:18 AM

I think I would describe myself a something between a lumper and a splitter.
-
I haven't studied any official courses in genealogy nor library science. However, I do have a couple university degrees, so I have had to learn to use the MLA method of documenting my educational papers. My favorite helper for this task is KnightCite (http://www.calvin.ed...tcite/index.php).
-
I agree with the definition of a source by RWells1938: Source: A supplier of information which answers the question "How Do I Know That?"
-
Early on in my research, I had a source called "Ancestry.com," and all info gleaned from the Ancestry online archives was lumped into that one source. As my database grew, that made less sense to me, and I now create a new source for each new Ancestry database, and use Ancestry.com as the Repository. I use only ONE template, "Book, Basic format," although I have many "Free-Form" sources that resulted from importing GEDCOMs of my earliest work (done in programs other than RM) and from other GEDCOMs imported from Ancestry. If the Ancestry database provides an image, I save that to a set of nested sub-folders on my harddrive and link to it as Media (almost always as a link to a fact, using the column with the camera icon on the Edit Person screen). I copy & paste the Ancestry index info into Detail text | Research Notes. Then I can memorize that citation and paste it multiple times to all of the individuals and facts that I add to my database that are mentioned/supported by this citation.
-
My main goal is to have a source citation which I can follow back to the original. A secondary goal is to avoid what I perceive as excessive complication engendered by using dozens of different templates. That means I don't always use the "Book, Basic format" template exactly as its designer intended. For example, in the "Publish Date" field I typically insert the date that the source was published, sometimes also including the date that I accessed the source (especially if the published date is unavailable). For printed books, I now add the ISBN to the published date field, after the date, in [square brackets]. For books referenced in a library, I add the call number into the "Call number" field for the repository. For webpages, I copy the URL into the Call number field.
-
So, as a lumper, I have one source for each census (e.g., 1930 US census). I create a single citation for a family group (even if the family group is spread over two pages in the original). This citation gets pasted to person in the family group: to General sources; to an Alternate Name; and to Census, Occupation, Birth, Immigration and other appropriate facts.
-
As a splitter, if I find two family groups on the same census page, I create separate citations for each family, and save the source image to two different filenames (since my file-naming scheme uses the head of family given-last-name and line number).
-
For newspaper articles, I add the newspaper's publisher as a Repository, then create a separate source for each article (using the article title as the source name). As with the census citations, the newspaper article citation will be pasted to multiple persons and facts. If the newspaper article is short, or already in text format, I paste it into the Research notes field for the source. The image, if available, gets linked to the key person and fact.
-
A current Narrative Report for 25 generations of my ancestors is 721 pages - 525 pages of narrative; 95 pages with 2246 endnotes; 47 pages of bibliography; and, 54 pages of name and place indices. I don't see any real importance to these numbers, except that the body of the narrative is the bulk of the report.
-
It has always bothered me that the narrative reports do NOT include a list of the repositories. After all, knowing where to find a source is essential to verifying that the source was cited correctly!

#12 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 8414 posts

Posted 10 April 2013 - 06:03 PM

Users can use Source Templates any way that works for them. But, as I read your story Jerry I wondered if the whole purpose of switching was to make it easier to change Citation Details later, could you not use "Find Everywhere" to locate those citation details you wanted to change and not edit them easily there?
Renee
RootsMagic

#13 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3534 posts

Posted 10 April 2013 - 10:32 PM

Users can use Source Templates any way that works for them. But, as I read your story Jerry I wondered if the whole purpose of switching was to make it easier to change Citation Details later, could you not use "Find Everywhere" to locate those citation details you wanted to change and not edit them easily there?


Short answer: no, not at all easily.

This is a real example, not fudged up at all to be associated with the mythical John Doe family. Do a Find Everywhere for the following string in my real, live, production database.

Wyatt M Nichols, Sr., The Nichols Book, Some Early Nichols Families in America, Their Origins and Histories (First Edition, 1962, To Be Copyrighted, Bloch Printing Company, Cleveland 5, Ohio), p.74

There are 21 matches and each of the matches reports that it is a match on a citation. Obviously, this is one of my old, lumped citations. If there is something in the Detail Text -> Research Notes area that needs to be changed, then I need to change it 21 times. If there is something in the Media area that needs to be changed, then I need to change it 21 times. In my new, extreme splitting style, any such change would only need to be made 1 time. Also, I could find the item to change much faster in the List->Sources process than the time required by Find Everywhere.

In addition, using Find Everywhere to try to address this problem has a sense of flying blind. For example, p.74 has information on many different people, and for any one particular person p.74 might be cited by several of that person's facts. I processed the information in The Nichols Book a long time ago - back in the FO days. Even though p.74 has information about many different people, I transcribed p.74 so that the Research Notes area for William Nichols only contains the information from p.74 about William Nichols and the Research Notes area for John Nichols only contains the information from p.74 about John Nichols, etc. for all the other people on p.74. But Find Everywhere simply tells me that there was a citation containing the character string I was searching for, and it does not tell me the name or record number of the person with the citation nor does it tell me the fact with the citation. Indeed, even though Find Everywhere is a live report, for citations it only shows me the citation itself and it exposes absolutely nothing about what the citation is associated with. With my extreme splitting strategy, I would solve the problem very simply by transcribing the entirety of p.74 into the Master Text -> Source Text area of the Master Source for p.74, and by connecting an image of p.74 to the Media area of the Master Source for p.74. Having done so, any required fixing becomes trivial.

Jerry

P.S. (and off subject) The author of the book in question died in 1965 and his widow died in 1987. I spoke with one of his nephews in the late 1990's who reported that the book was never copyrighted, and that after his widow's death in 1987 the children simply discarded his manuscript along with all of his research materials. Sadly, these kinds of losses of a lifetime of research happen all the time.

#14 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 8414 posts

Posted 11 April 2013 - 09:33 AM

Interesting. With your comments about page 74 it makes me realize that if we did have the ability to allow massive edits on the source citation level that it could potentially cause more damage than help.
Renee
RootsMagic

#15 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6218 posts

Posted 11 April 2013 - 11:52 AM

Interesting. With your comments about page 74 it makes me realize that if we did have the ability to allow massive edits on the source citation level that it could potentially cause more damage than help.

Powerful tools always increase risk but that is no argument against providing them. The need remains. What is needed is a Shared Citation, similar in concept to a Shared Event. Jerry's been radicalized to extreme measures because there is no Shared Citation short of the ultimate split, where every different citation is itself a Master Source.

I have to wonder, Jerry, whether there is not the possibility in your concept of using the Research Notes and Citation Comments (at the Source Detail level) to some advantage while keeping all Footnote, Short Footnote and Bibliography sentence template fields in the Master Source. Of course, the quality is dependent on how the citation is used so that remains a per event edit.

I think there might even be the possibility of having fields in Source Details that are not part of any sentence template to some advantage without jeopardizing the Short Footnote and Bibliography. IIRC from looking at Jeff La Marca's "Simplified Citations – Census", even non-sentence Source Details fields are exported to the PAGE tag which may or may not be desirable. Non-sentence Master Source fields are not exported; only those in the Footnote sentence are.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#16 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3534 posts

Posted 19 April 2013 - 01:54 PM

The need remains. What is needed is a Shared Citation, similar in concept to a Shared Event.


Shared Events have been my analogy as well, but I wonder if a better analogy might be Place Details. The combination of a Place and a Place Detail is required to print out a "complete place" in a report. Similarly, if sources worked more like places then the combination of a Source and a Source Detail would be required to print out a "complete citation" in a report. Well, to a certain extent that's the way a Source (called Master Source) and a Source Detail already work. But it's not quite the same as Place and Place Detail. So I continue in the next paragraphs about how Source and Source Detail could be made to work a little more like Place and Place Detail.

Each Place Detail is subsidiary to a Place. Changing a Place Detail in the Place List changes the Place Detail everywhere that the Place/Place Detail combination appears. Similarly, RM could be changed so that each Source Detail would be subsidiary to a Source. Then changing a Source Detail in the Source List would change the Source Detail everywhere that that particular Source/Source Detail appears in a citation. As I said, it really doesn't work that way now and that's the change that would need to be made to solve the problem.

It's probably worth looking at the user interface for Lists->Place List and Lists->Source List in a little more detail better to see where the problem is for Lists->Source Lists.
  • The screen for Lists->Place List has three panes. The left hand pane is the list of places which is scrollable, plus you can type in the name of place to go directly to a place. There are panes throughout RM that work this way, and they are fine. The upper right hand pane contains the rest of the information about the place besides the name - things like the Standardized Place Name, the Abbreviated Place Name, the geocoding for the place name, and the note for the place name. The lower right hand pane contains a list of Place Details for the Place, and is directly clickable to open and edit any of the Place Detail Items. This is a very clean and simple user interface that does not include the excessive clickiness and excessive drill downs to see otherwise invisible data that I complain about so much in some parts of RM.
  • The screen for Lists->Source List has only two panes. The left hand pane works just like the left hand pane for places. The right hand pane works just like the upper right hand pane for places. What is missing, and what would solve the problem, is the lower right hand pane which could and should show a list of all the Source Detail items associated with the Source. I could go on and on with further details of how this would all work, even down to how the underlying Source Table and Citation Table would have to be changed to make this work. But I think describing the analogy at this level of detail is adequate to describe how it would work.
One more piece of the puzzle needs to be described, namely how a user would select these Source/Source Detail combinations or make new ones. Right now, you can basically "Add new source" or "Cite new source". A second step would need to be added to each where you could "Add new source detail" or "cite existing source detail". The exciting option here would be the "cite existing source detail" option which would totally solve my problem of having to find and fix every instance of an existing source detail as a totally independent operation. Again, there are a number of other minor (or maybe even major) tweaks that would have to be made for this to work correctly. But if done well, it could be a humongous improvement to the process of entering and maintaining citations.

Finally (and I know this message is too long already), I feel obliged to suggest how the Memorize and Paste function would work for citations in this new world. From the user's perspective, the initial Memorize and Paste would look just like it does now. The difference behind the scenes is that Memorize and Paste of citations would really just be reusing the same citation rather than creating a whole new one the way it does now. The key difference that the user would see is that from the Edit Person screen, changing a citation would appear to the user to be changing just that once instance of the citation (behind the scenes it would be making a new citation). Then from List->Sources, changing a citation would be changing all the instances of the citation. This is the critical functionality that is presently missing. And back to the analogy with Place/Place Details, it's exactly how the user interface works with places.

Jerry

#17 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 8414 posts

Posted 22 April 2013 - 11:19 AM

So I'm thinking out loud about this. We would still have a problem with people that cite page 74. What if some really are on that page and others you find later are given that incorrectly. Would you massively change everyone to a new page number because they are linked to that citation? What if a person is on page 74, 106 and 312? Do you have citations for every page they could be on instead of using one citation to list them all? We have to look at all the potential ramifications to any changes before we ever implement a change that can destroy or effect other data for people in the database.

Right now I still see "Find Everywhere" as the best solution for locating people that need a change made on their source citation. Maybe some improvements on how to locate those citations could be made. My gut tells me that massive changes on the citation level for all uses of it can cause to much damage.
Renee
RootsMagic

#18 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6218 posts

Posted 22 April 2013 - 11:44 AM

I think Jerry has thought through this process very well. In his last paragraph, he distinguished between an edit that affects all uses of a common citation and one that affects only one use of it (I.e., replaces that one use with a new, different citation of the same source). What is important is that any edit of a shared event, multiply used place or place detail, common source or the proposed common citation should be accompanied by an information message indicating the number of other uses of the item that will be affected plus an optional list of the affected uses. The informed user is a safe user when allowed to back out of an operation she realizes is more sweeping than expected.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#19 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3534 posts

Posted 22 April 2013 - 09:41 PM

What is important is that any edit of .................. the proposed common citation should be accompanied by an information message indicating the number of other uses of the item that will be affected plus an optional list of the affected uses.


The point raised by Renee about my p.74 example and Tom's response are right on point. However, I do think I have outlined an adequate solution.

Let me respond first with an example from the history of Family Origins and RootsMagic. When I first started using Family Origins, there of course was no Place Details feature. But events certainly had places and there was certainly a Lists->Place Lists function. Suppose I had a fact for John Doe that said he died in 1895 in Tennessee, based on the evidence then in hand. And then suppose that new evidence was uncovered that John Doe died in 1895 in Missouri. I would go into John's death fact to change Tennessee to Missouri, and I would get a message warning me that I was about to change every occurrence of Tennessee in my database to Missouri. There was an option along with the warning allowing me to change the occurrence of Tennessee to Missouri just for John's death fact. If that's the option I chose, then all would be well. But if by accident or lack of understanding of the message I chose the other option, then Family Origins would indeed change all occurrences of Tennessee to Missouri. There would be no graceful way to recover from the error, and I would have to go back to my most recent backup of Family Origins.

Somewhere along the way in the late FO days or the early RM days, the behavior of the program was changed so that a change of a Place in a fact only changed the place for that fact and there was no message. A global change to a Place thereafter could only be accomplished only from the Place List. This new behavior was a very good thing. Later on when Place Details came along, the behavior of changing Place Details was treated the same as for Places. Changing a Place Detail for a fact would change the Place Detail only for that fact, and changing a Place Detail from the Place List would change the Place Detail. This is precisely the behavior I'm suggesting for Source and Source Details.

As to Renee's concern about my example of p.74 in The Nichols Book, it would be a non-issue for a newly added and extremely split Source. I would transcribe as much or little of the actual p.74 into the Master Text field of the Master Source as I needed. If I were only using the Source for one or two individuals from the page, I would probably only transcribe the information for those one or two individuals. Otherwise, I would transcribe the whole page. I could change my mind and transcribe more of the page at any point without having to go back and redo anything else. And of course the image of p.74 would be linked to the Master Source for the extremely split Master Source for p.74. The only problem would be if I wanted to convert an old style lumped source for The Nichols Book where the p.74 information appeared as a part of the Source Details to a new style split source for The Nichols Book where the p.74 information appeared as a part of the Master Source. In this case, I wouldn't want an automated procedure for the conversion. I would do it by hand.

Finally, nobody has really mentioned the most problematic parts of my suggestion. Namely, for both the Place and the Place Details you have sort of a "what you see is what you get" situation. But in the case of Source and Source Details, what you see is dependent upon the Source Template and the text that you see can involve an intermixture of the text contributed by the Source and the Source Details. But I think these problems can be overcome. In the meantime, I'll continue splitting sources as I've described in this thread.

Jerry

#20 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 8414 posts

Posted 23 April 2013 - 08:49 AM

Confirming enhancement request is in our tracking system.
Renee
RootsMagic