Jump to content


Photo

Adventures in Extreme Splitting


  • Please log in to reply
66 replies to this topic

#41 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3590 posts

Posted 03 July 2013 - 10:05 PM

I have Person sources now being named oldmastersourcename: personname (Person) with the intention of extending that pattern to oldmastersourcename: personname (eventname). I'm not sure how well that will work from a user perspective but it is programmatically appealing. Any comment?


I would probably have to see some examples to be able to determine what seems to work and what doesn't. When understanding how pieces of things fit together to make a whole, I think most people either start with a general pattern and work their way to specific examples, or else start with specific examples and work their way to a general pattern. I find that I have to work the problem from both directions, more or less at the same time. The general rules feed off the examples, but the examples also feed of the general rules in a feedback loop.

I do like your use of the colon. I'm kind of a stickler about certain kinds of punctuation issues, and I often find myself at odds with the English teachers and English professors of the world when it comes to punctuation. Here's just one of many examples. Suppose you have a citation for a book that's of the form "author, title" without the quotes. A real citation of a book ought to contain some additional items such as publication date and place etc., but to keep things simple let's just restrict ourselves to the author and title. So we might have something like "John Doe, The Doe Family of Boondock County" without the quotes - or maybe depending on the citation authority the title needs to be in italics or underlined or in bold or some such, viz. "John Doe, The Doe Family of Boondock County" without the quotes. But I'm not so worried about the italics as I am the comma. Here's what happens if the author's name becomes inverted for proper alphabetization in a bibliography: "Doe, John, The Doe Family of Boondock County" without the quotes.

There is a quote from the famous mathematician John Von Neumann which can be repurposed slightly to convey how upset I am with this bibliographic entry. The quote is

Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin.


Whether you know anything about arithmetical methods of producing random digits is beside the point. Von Neumann's point is that if you are producing random digits in this manner you shouldn't be, to the extent that you should be considered to be in a state of sin if you do so (with a huge tongue in cheek, of course). Similarly, if you produce a bibliographic entry such as "Doe, John, The Doe Family of Boondock County" without the quotes then you are so abusing the poor comma that you must therefore be considered in a state of sin. :)

Which is to say that the first comma has one purpose and the second comma has a different purpose. In fact, it looks like you have a list of three items: 1) Doe, 2) John, and 3) The Doe Family of Boondock County, when in fact Doe and John are a part of the same item. So my naming conventions for Master Source names and my footnote sentences and bibliography sentences use a semi-colon to solve the problem, viz., "John Doe; The Doe Family of Boondock County" without the quotes and "Doe, John; The Doe Family of Boondock County" without the quotes, respectively. Strictly speaking, the footnote form doesn't need the semi-colon and the bibliographic form does, but I think they both have to use the semi-colon in order to make any consistent sense.

Jerry



(Added 4/7/2013)

I just checked Evidence Explained p.77. It appears that I'm much more in accord with Elizabeth Shown Mills on this comma issue than I expected. I quote: "Semicolons are used to mark the major divisions. Within each of those units, commas separate the smaller parts."

But to me, this still begs the questions of extra commas that you get, for example, when you invert first and last name for alphabetization or when there is a comma before a suffix such as Jr. or Sr. Those extra commas still can throw the whole universe out of balance.

#42 LeeHollenbeck

LeeHollenbeck

    Member

  • Members
  • PipPip
  • 23 posts

Posted 27 July 2013 - 02:49 PM

Hi Jerry,

Thank you so much for presenting your exploration of extreme splitting. I especially liked your SnagIt demo, which demonstrated extreme splitting in action.

A few years ago I decided that my database was such a mess that I would start over from scratch and enter everything into a new database source by source.

I had been using Brother's Keeper, but did some research and decided to go with RootsMagic 4.

I have some experience with genealogy and wish to figure out how to best enter information before entering everything so perhaps I will not feel I need to re-enter everything again. I have played around with RootsMagic off and on the last few years, creating small databases, playing with RootsMagic features, and seeing how it prints out in various reports, especially narraitve reports. With FamilySearch Family Tree approaching, I was reluctant to do much without first examining thoroughly FS Family Tree.

I was probably middle of the lumper/splitter road in entering sources and was very disgruntled with all the redundancy (still am.) Even if I did not make so many typos, I would still be editing the sources/citations frequently because I like to tweak things. Finding all the citations and doing the copy and pastes was driving me crazy.

Especially after playing with FS Family Tree, I decided to become a hearty spliter. I've been looking in the forums and mailing lists for information on how others are handling the redundancy. and what problems and solutions they have found.

I came across your thread here and had to respond about how much I appreciate you explaining why you have become an extreme splitter and demonstrating doing so.

What it came down to for me, is "What is a citation?" In my citations, I want the Source Details to explain why this source supports this fact/event. I include in Research Notes and Detail Comments information about this source in regards to this fact/event. Everything else about the source should be not in Source Details. This made me decide to become an hearty splitter.

The idea of having a separate master source for every census household, every obituary , every bio, every vital record, and every tombstone seemed daunting, but you have made it seem less so.

What I really want is for there to be three levels availble in sources. Instead of just having two levels, Master source and Citation Detail, I would very much like to also have a middle table - Source Item. Splitters and lumpers could still decide what they want to put where, but there would be less redundancy, less copy and paste, fewer places to edit the same information. if someone did not want to use the new table, they would not have to. As you pointed out with your first example of a book, there are some sources which splitters and lumpers might treat the same, and the middle table would not be useful for such a source. However most of my sources are things that are one item of many in a source, an item more a whole thing itself than a page. The item usually has one person or family as its principal subject, and maybe even one main fact or event, but the item supports and is cited for many facts/events. Some sources could use far more than three tables, but three would be better for them than two.

I see my own adventure in extreme lumping as one step in my quest for a middle table.

#43 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6254 posts

Posted 27 July 2013 - 09:29 PM

At the wiki page Sources - Adventures in Extreme Splitting, you will find my just completed heavy script that converts "lumpy" sources from all template types and Free Form to "ultimately split". This is an invitation to the adventuresome to apply it to a copy of your database and see how it produces better GEDCOM output from most templated sources, more complete transfers of sources to FamilySearch Family Tree and other benefits as Jerry has espoused. Undoubtedly, there will be some disadvantages (and maybe some bugs, too) but only you can judge what works best for your needs.

I have been working on a script that converts citations of lumped sources to what I might call "ultimately split sources" (possibly going beyond "extremely split"!) and am almost finished but for naming the new sources. I have Person sources now being named oldmastersourcename: personname (Person) with the intention of extending that pattern to oldmastersourcename: personname (eventname). I'm not sure how well that will work from a user perspective but it is programmatically appealing. Any comment?

The goal for this script is to make it easy to transform an existing database to one with all sources ultimately split so that anyone (with SQLIte basics) can see how this affects outputs from their database. I'm expecting to see GEDCOM source output that matches report endnotes, regardless of which source template is used.


Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#44 LeeHollenbeck

LeeHollenbeck

    Member

  • Members
  • PipPip
  • 23 posts

Posted 28 July 2013 - 10:18 AM

Also, Jerry, in your reply to Renee, you give a good example of what I most frequently find frustrating -- the editing of Detail Text/Comments. This is what I am usually fixing, not the sourcing fields. I know that it would be very difficult to put a source item table into RootsMagic, so I thought a clip collection would address much of my problem that I was sure many others were having. If I could have <Doe, John 1833-1911 - Obit> be replaced by the same text on export or printing everywhere in the database, my edits would be much reduced.

This proposed clip collection would grow very long and it does not fix the problem of attaching media or the other problems you mention. Instead of wishing for a very long clip list, it looks like I am going to have a very long master source list.

I still like the page field in details. If a source item extends over more than one page, I would want just one master source, but I would also like to indicate on which page a phrase supporting a particular fact resides. Mabey this is just a lumping tendancy rearing its head. Perhaps I should put the page numpber with the quoted phrase in Detail Text? That could look a little ugly. I probably would already have something like "pages 72-73" in the Master Source for the Full Footnote.

I still have to work this out before I go whole hog and commit. If the section of text of such interest on p72 of Some Early Nichols Families in America, Their Origins and Histories extended in some small part onto page 73, what would you do?

#45 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3590 posts

Posted 28 July 2013 - 08:15 PM

I still have to work this out before I go whole hog and commit. If the section of text of such interest on p72 of Some Early Nichols Families in America, Their Origins and Histories extended in some small part onto page 73, what would you do?


I'm still making up some of this as I go along. One possibility that comes to mind is to keep page 72 and page 73 as separate Master Sources and simply cite both Master Sources for the same person or for the same fact as appropriate. That's neat and clean, but it might look a little funny in a listing of footnotes or endnotes. The other possibility would be that page 72 would be a Master source, page 73 would be a Master Source, and pages 72-73 would be a third Master Source, again cited as appropriate. I would have to look at a few examples in reports to decide which way to go.

Also, it occurs to me that the same problem really exists if you keep things like media files and notes of various sorts associated with Source Details. For example, what if you have an image of page 72 associated with Source Details and a transcription of page 72 associated with Source Details in the Research Notes or Comments field. And what if similarly you have the same information for page 73 associated with a different set of Source Details under the same Master Source. In this case, how would you handle the media files and notes for something that spans pages 72 and 73?

Jerry

#46 Laura

Laura

    Advanced Member

  • Members
  • PipPipPip
  • 4276 posts

Posted 28 July 2013 - 10:27 PM

As it makes no difference how many times a media is linked, I don't see any problem.

Master Source, p. 72, link media for p. 72
Master Source, p. 73, link media for p. 73
Master Source, p. 72-73, link media for both p. 73 and p. 73

If p. 72 and p. 73 are in one media file, link that one media file to all three Master Sources.

The same with Source details for each page.

And the media can be linked to both a Master source and the Source detail if desired.

The choice of where to link media is the same whether we are splitting or lumping sources.

#47 LeeHollenbeck

LeeHollenbeck

    Member

  • Members
  • PipPip
  • 23 posts

Posted 29 July 2013 - 01:01 PM

Jerry, you make avery good point about the notes and media. I have not put many photos into my genealogy program, so my mind usually thinks of the text. I do have a lot of media to put in the database, so I do really need keep media in mind. I am also still making this up as a go along, so it is wonderful to see what you are doing and what problems and solutions you are finding.

I do have some idea of what I am doing ;-) because I had previously, where applicable, put transcriptions of what I consider Source Items in the Detail Research Notes, and I put in Detail Comments sometimes scrap of text which I am now putting in Detail Research Notes. It had very much bothered me how I was using the Master Source and Source Detail memo fields because I was usually lumping my sources too much to put much of anything in the Master Source memo fields and I was cramming things into Detail Research Notes and using Detail Comments for both information about the supporting text and my occassional thought (comment) in regards to this text. Seeing how Family Tree was working was the final straw which made me become much more splitty.

I guess I am stopping one short of extreme splitting. Instead of making a master source for every citation, I am making a master source for what in my mind is a Source Item. Although I am not Extreme Splitting, I am MUCH more splitty than I used to be. That "what in my mind is a Source Item" is key. To me a Source Item (as opposed to a Source, which I consider to be as somthing is normally specified in the bibliography) is an obituary, a tombstone, a census household, ect. The type of things I am putting in Detail Research Notes are scraps of text specific to the citation such as "John, a native of Kentucky". This is just to underscore what in particular I am reading in the source to support the fact/event. At most I might have a couple sentences, but that is rare, and often I put nothing. Now, thinking about it, I bet nearly every time I have used the Page field in in my new splitty sources, I probably also had something in the Detail Research Notes. I could go back and append the page number in the Detail Research Notes if I thought a more specific location than that given in my splitty source was really neccessary. If I am printing Detail Research Notes, the footnote/endnote could look like:
... pages 72-73. "John, a native of Kentucky" (page 73, paragraph 2).

One of the problems I am trying to solve is to not have to edit many citations if I want to change something in a transcription of, for example, an obituary. If I go to Extreme Splitting, creating a master source for every refernce to a source, I would then have to edit many Master Sources. I believe it would easier to edit a bunch of Master Sources rather than Source Details, especially if I name the source titles such that they show up in the Master Source List next to each other. However, I would still be making a lot of copy and pastes. The Source Item, especially transcriptions in the Source Item, is what I am usually fixing.

I think Laura is right in that where to put the media is a choice of lumping and splitting. Thinking of what to have for the media is probably a very good method for determining what I would want as a Source Item. Whether it is appending images of different papers for a Marriage Record or appending a couple of pages for a section of a book, what a Source Item is will depend on what a researcher chooses to lump and split.

Perhaps besides a question of lumping and splitting, this also involves a question of terminology. When someone (even myself) says "Citation" I am often unsure of the granularity meant by the term.

When I assign a source, I almost always assign a source to a fact/event. Only rarely do I assign a source to a person or family. For example, if the only thing I know about a person is that the person is a parent to someone in the database, I would assign the source that mentions this to the child or the parent. I suppose I consider this Person an "existence" event or a name fact. This makes what I am recording in the Source Details (Citation) table what I consider to be specific to the relationship between this fact and this source.

When Shared Citations is brought up, I recoil. Currently, to me, it looks like we have sources split between two tables, sources and citations (source details). So I assume what is being meant by Shared Citations is having an entry in the citation table, as it currently is being used by someone using the source templates, being shared amongst many persons/families/facts. Since I think of the table as something which should hold only the information specific to the source and one fact, I shrudder.

However, if what some people mean by Shared Citations is:
1) to have what is in the Master Source, plus all those fields in the Source Details section of source templates, plus the Source Details Research Notes and Comments, plus Source Details Media be what is to be shared;
2) and to have the link between all this "Citation" and the fact/person/family include a memo field or two to explain why this "Citation" supports this fact, and include the Quality ratings,
Well, then I'm totally on board. This is adding my middle table, only instead of looking at it as adding a middle table, what is added is a bottom table.

So if what is meant by a "Citation" is the above, I am an Extreme Splitter.

#48 Laura

Laura

    Advanced Member

  • Members
  • PipPipPip
  • 4276 posts

Posted 29 July 2013 - 04:15 PM

In RM, Master Source + Source details = Source Citation. That is, unless you are extreme spitting with no Source details, then it is Master Source = Source citation. :)

I put the transcriptions for a source in the Source detail, Research note along with any comments I want to share with someone else in a report. I put comments I don't want to share in a report in the Source detail Comments.

If a user is going to be an extreme splitter and not create Source details, put those transcriptions in the fact note if you want them printed. You can enclose them in curly brackets in the note if you want to choose whether or not they are printed in a report or exported in a gedcom. If you don't want the transcriptions printed in a report, enter them in the Master Source text.

While this discussion is interesting and informative, I am concerned that it is making adding sources to RM seem difficult and more of a chore than it really is.

Create a new database and drag and drop a few people into it. Play around with how you want to add Master sources and Source detail if any and how they will look when printed. Decide where you want to enter transcriptions and notes and link media.

When you know how you want to do sources, then you will know how you want to incorporate that procedure in your main database.

I created a Source list report with All sources selected and Citation details, Research notes, and Comments selected to see if I might want to change my mind and become an extreme splitter to avoid having to correct Source details occasionally. I saved it to a file and then opened the file.

My report with those choices is 3893 pages. With just the Master source names, the report is 47 pages. I didn't do any averaging of how many Source details a Master source had.

I did determine that it would be way more work to split my Master sources than I would ever do correcting Source details.

#49 LeeHollenbeck

LeeHollenbeck

    Member

  • Members
  • PipPip
  • 23 posts

Posted 30 July 2013 - 12:47 AM

Hi Laura,

Thank you for clearing up the citation question. :D

I very much like hearing exactly how other people use RM and why they use it the way they do. I'm alway looking for a better way of getting the results I want.

I certainly would not want someone new to RootsMagic to think it was any more difficult to enter sources in RootsMagic than in another program. Between the many source templates and the free form template, it is a simple thing to install the program and start entering perfectly acceptable sources. There are MANY templates, but they are labled well, usually handle blank fields well, and after a bit of entering, a user will know which ones to make favorites. It also is very versatile and customizable. Taking advatage of this versatility is completely optional. A user does not have to customize anything unless they are interested in doing so. It can be used very simply. I love that a user can choose to keep it simple or do some powerful customization with data entry, organization, reports, and searches. This, and the excellent support and forums, is why I chose RootsMagic after trying out the competitors' programs. And, since I bought RootsMagic4, it has kept gettting better and better. I am especially impressed with how well it interfaces with FS Family Tree. Fabulous!!

That said, now that I have used the program for some time, I am looking to see how I can best use it for me. If I was not starting over, I don't know that I would consider radically changing how I enter future sources, much less go back and redo the old ones. When I first installed RM 4, my old Brother's Keeper database was such a mess (entirely my fault) that I decided to re-enter everthing into the new database and do it right. This is my 2nd attempt with RootsMagic on that. (Decision to redo database second time also due totally my lack of organization.)

I had previously been making the lumper/splitter line roughly the same as that in the provided source templates. For the Master Source I would use census records information down to the county level, general information about newspaper titles, cemeteries, collections of records, and books. In the Source details fields I had census record info from Civil Division to person of interest, newspaper articles, grave markers, records, and pages.

I do not want to be an extreme splitter. If I wished to edit or add something to a newspaper, where, with so many, many, many articles, would I put this remark about the newspaper? I like to keep things in the database. I want order. Would I have to check to see if I put info about the newspaper in some odd place in the database like a research log or a person? or maybe even somewhere external to the database? Ack! I'd be taking a look at a SQLite manager for simple edits. I very much do NOT want to be an extreme splitter.

But, on the other hand, what if I want to edit an obituary? - one of those great long obituaries which start off with "Pioneer Citizen ... "? I love having transcriptions, and I often need to edit them. Should I have an entire transcription of the obituary in Source Detail Research Notes of all citations of the obituary? I did that for some time. (Memorize works precisely as it should, and it works wonderfully well when first attaching a source to many facts.) In my database it was the Source Detail Research Notes which was the field I did not print. Unfortunately, I make many typos and am very poor at seeing them. Sometimes I make an edit, even though everything is technically okay, because I want to make it better than okay. I make many, many edits. It is how I work. Eventually, I decided to have the transcription in one fact/person/family be the "good" copy and try to remember where the "good" copy existed. This is sometimes bothersome because where I would put the good copy is not always as clear cut as with an obituary.

With the obituary example, I suppose I could make the large articles which are cited many times into Master Sources. The Newspaper itself could be an additonal Master Source, and Newspaper items which support few facts could use the Newspaper Master Source. Even if a detail has to be edited in only a few locations, having to edit the same thing multiple times gets really old. It is also prone to error. Also, if the thing I need to edit about the Newspaper is something also in the citation of all those Master Source Newspaper Articles, I still have many Master Sources to edit.

I don't even want to talk about censi.

The real problem is 1) the source description and information itself often needs at least two layers of hierarchy to be managable, and 2) the details about how the source relates to the person/family/fact needs to be separated from information regarding only the source. From what I see, the interest in extreme splitting is really a need for details about the source to be separated from the relationship of the source to the person/family/fact.

In Family Tree there is 1) the citation, 2) the reason the source is being attached to the person/family, and 3) the person/family. Other than a note field for whatever the user thinks is neccessary, and some tags for major facts/events, that's it in Family Tree for citing sources. If you are using FamilySearch Sources, they create and promise to maintain the correctness of the citations for you. This is essentially what we have in extreme splitting - Citation, relationship of citation to Fact/Person/Family, and Fact/Person/Family. Problem is, in RootsMagic (no other genealogy program I looked at did any better at this than RootsMagic, and I found RootsMagic to have the easiest sourcing to use), we are also managing our source description and information ourselves and we need more than the one table available for this in extreme splitting.

Sorry for getting all naked here, but all this redundancy is insane. I know if this was easy to do, it would already be done. There are very smart people working on this software. maybe the real problem is I make a LOT of typos.

#50 Pheney

Pheney

    Advanced Member

  • Members
  • PipPipPip
  • 31 posts

Posted 15 August 2013 - 05:58 PM

I have been following this discussion on splitting of sources with interest, and have also had a look at the SQlite wiki pages on using SQL to try to address the issues. My interests are basic – researching, recording and preserving family history with “preserving” including the ability to transfer information to other people and websites accurately and with no loss of content – the transfer part being idealistic, I know. I have become an extreme (possibly ultimate) splitter for many of the reasons outlined in this discussion.

In this note, I assume that gedcom is the standard transfer method, and that GedcomX does not address the issues.

I have used Rootsmagic (RM) for some years, but generally use only the basic features, mainly in an attempt to make data transfer easier. For instance, I have very few shared events, only recently started added media to sources, and do not use web tags.

I will attempt to use words with their meanings in RM, such as:
  • Source List: the list of Master Source types (templates);
  • Master Source: a source in the list of user-created master sources developed from the Source List;
  • Source: a user-created source attached to a person in a RM database. It includes text, comments, media, quality etc;
  • Source Details fields – any fields, specific to a Master Source, which are added to a Master Source in the creation of a Source;
  • Page Number field – the only Source Detail field available in a Master Source derived from the freeform template in the Source List;
  • Source citation – the footnote, short footnote and bibliography associated with a Source.

My key concerns revolve around template-based Source citation footnotes which do not transfer properly via gedcom; these appear to be Sources that have Source Details field/s.
I prefer to work entirely within RM, but admire the efforts at producing SQlite queries to convert existing source citations to a form that transfers properly via gedcom.

After numerous attempts at producing source citations in RM, I have developed a slow, clunky method that produces acceptable results – probably close to “ultimate splitting”. This is:
  • prepare a draft Master Source using an appropriate template from the Source List;
  • use the draft Master Source to make a Source i.e. the Master Source plus Source Details fields;
  • copy and paste the footnote, short footnote and bibliography of the new Source into a blank freeform Source. The Page Number field is not used;
  • the draft Master Source can either be deleted, or retained to develop Sources with different Source Details fields. Retaining it causes the number of Master Sources to increase dramatically; deleting it causes issues with creating similar Sources – to start from scratch, or to copy and edit an appropriate freeform Master Source? Copying risks omitting fields which were in the draft Master Source, but were not used. Currently I favour the delete and copy option, on the basis that similar Sources will use the same fields.

This process clearly has drawbacks, such as the time and effort involved, and the proliferation of Master Sources. The latter appears to be unavoidable in this “ultimate splitting” type of approach.

Several solutions come to mind, bearing in mind my limited use of RM features, so I may have overlooked some issues:
  • a utility such as RMtrix which converts a template-based Source into freeform, without using the page number field. Such a utility would need to come with full instructions for those of us who are not even sure what a SQlite query is;
  • inclusion in RM of the ability to convert a template Source to a freeform Source – either at creation or later. This could include conversion of multiple template Sources in a batch;
  • no change to the RM Source Manager, but a change in the way that RM produces gedcom files – so that the output replicates the footnote part of the citation, with the page number field not used. From my limited experience with the RM – FSFT interface, such footnotes appear to transfer properly from RM to FSFT.

A couple of comments

The above assumes transfer by way of gedcom (and its limited capabilities). Any solution should, as much as possible, be future-proofed by avoiding RM-specific features and taking into account GedcomX and other possible transfer mechanisms (if this is feasible). Short footnotes and bibliographies remain non-transferable – at present, at least.

I use only basic Source Manager features in RM – very few source text, source comments, research notes, detail comments, and media quality entries and no and web tags. These features would need to be considered - as other people in this discussion have done.

MY OVERALL PREFERENCE is to use RM templates to develop Master Sources as at present, including the Source Details field, but with the ability to convert each Source to freeform, with each Source becoming a Master Source. This could mean multiple freeform Master Sources from each template Master Source – one for each different set of detail fields.

#51 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6254 posts

Posted 15 August 2013 - 09:54 PM

The latest webinar Sharing Sources Between RootsMagic and FamilySearch Family Tree uses an "ultimately split" source as an example in transferring from RootsMagic to FamilySearch Family Tree because the Research Note and WebTag for a conventional source cannot be transferred. Any source on FSFT transferred to RootsMagic is inherently also "ultimately split" in that there is nothing delivered into any of the Source Details fields, Text, or WebTags.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#52 Miranda

Miranda

    Advanced Member

  • Members
  • PipPipPip
  • 40 posts

Posted 17 August 2013 - 05:20 PM

But, on the other hand, what if I want to edit an obituary? - one of those great long obituaries which start off with "Pioneer Citizen ... "? I love having transcriptions, and I often need to edit them. Should I have an entire transcription of the obituary in Source Detail Research Notes of all citations of the obituary? I did that for some time. (Memorize works precisely as it should, and it works wonderfully well when first attaching a source to many facts.) In my database it was the Source Detail Research Notes which was the field I did not print. Unfortunately, I make many typos and am very poor at seeing them. Sometimes I make an edit, even though everything is technically okay, because I want to make it better than okay. I make many, many edits. It is how I work. Eventually, I decided to have the transcription in one fact/person/family be the "good" copy and try to remember where the "good" copy existed. This is sometimes bothersome because where I would put the good copy is not always as clear cut as with an obituary.

You are not alone.

I have been on what has turned into a 1.5 year hiatus from doing genealogy because the data entry is so cumbersome. I also make many edits. I just change my mind a lot about how I want things transcribed, which leads to me having to find all the sources over and over and over and over and making many many many edits just to change one thing.

#53 Pheney

Pheney

    Advanced Member

  • Members
  • PipPipPip
  • 31 posts

Posted 03 September 2013 - 01:03 AM

I have experimented, with some success, with the source splitting process outlined on the SQlite wiki pages - thanks, Tom. Some of the sources are freeform, others use templates with/without detail fields.

When the new database is opened in Rootsmagic, most citations show up properly. I had an issue with sources based on the "Obituary/Newspaper item" template. In some cases, the detail fields are misplaced, in others the detail fields move to Source Text and Research Notes.

When the new database is transferred via gedcom to Family Tree Maker or ancestry.com, there are also blank spaces in the footnote every 60-70 spaces, but this seems to happen for all citations, so may be a gedcom issue.

I use only a few of the 400+ source templates, so can't vouch for them all.

If the issue with the Obituary template can be fixed, the SQlite splitting process seems to give a way to export source citations. A pity it can't be done within Rootsmagic, at present. Here's hoping ...

#54 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6254 posts

Posted 03 September 2013 - 09:21 AM

I have experimented, with some success, with the source splitting process outlined on the SQlite wiki pages - thanks, Tom. Some of the sources are freeform, others use templates with/without detail fields.

When the new database is opened in Rootsmagic, most citations show up properly. I had an issue with sources based on the "Obituary/Newspaper item" template. In some cases, the detail fields are misplaced, in others the detail fields move to Source Text and Research Notes.

When the new database is transferred via gedcom to Family Tree Maker or ancestry.com, there are also blank spaces in the footnote every 60-70 spaces, but this seems to happen for all citations, so may be a gedcom issue.

I use only a few of the 400+ source templates, so can't vouch for them all.

If the issue with the Obituary template can be fixed, the SQlite splitting process seems to give a way to export source citations. A pity it can't be done within Rootsmagic, at present. Here's hoping ...

Thanks for the feedback, Pheney.

I don't understand why the Obituary/Newspaper template would misbehave if other templates do not - maybe something corrupted in your database, especially because detail fields are popping into places they have no connection with. Perhaps you can send me a copy of your database or an extract thereof that also misbehaves so I can look into it. Unfortunately, I will be bereft of tools for a little while starting soon. Is it a problem with all Obituary/Newspaper sources or just some? We can continue this privately.

The blank spaces showing up in FTM and Ancestry.com Family Trees must be a GEDCOM export/import incompatibility. I have seen unwanted word concatenation in the opposite direction!

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#55 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3421 posts

Posted 03 September 2013 - 10:56 AM

Guess I have always been a lumper and need to read and understand a lot more before driving a reconciliation on my Sources, which shamefully has never been a priority for me.

We are all limited by our visions and abilities

Whilst we can borrow from the visions of others we cannot always deliver.

 

User of Family Historian 6.2.7, Rootsmagic 7.6.2, Family Tree Maker 2014 & Legacy 7.5

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#56 Pheney

Pheney

    Advanced Member

  • Members
  • PipPipPip
  • 31 posts

Posted 06 September 2013 - 11:15 PM

I have experimented, with some success, with the source splitting process outlined on the SQlite wiki pages - thanks, Tom. Some of the sources are freeform, others use templates with/without detail fields.

When the new database is opened in Rootsmagic, most citations show up properly. I had an issue with sources based on the "Obituary/Newspaper item" template. In some cases, the detail fields are misplaced, in others the detail fields move to Source Text and Research Notes.

When the new database is transferred via gedcom to Family Tree Maker or ancestry.com, there are also blank spaces in the footnote every 60-70 spaces, but this seems to happen for all citations, so may be a gedcom issue.

I use only a few of the 400+ source templates, so can't vouch for them all.

If the issue with the Obituary template can be fixed, the SQlite splitting process seems to give a way to export source citations. A pity it can't be done within Rootsmagic, at present. Here's hoping ...

Further to my issue with the Obituary/Newspaper source template when using the splitting process - the problem is fixed, and I can now export via gedcom with all footnotes correct - apart from a few blank spaces that are not caused by the splitting process.
The secret is to make a copy (for splitting) of part or all of the RM database by drag-and-drop to an empty database, or by doing a full copy of the database. Not via a gedcom export/import ....
TomH has done a great job in crafting the SQL scripts, from my point of view they have made possible what was practically impossible without converting (to freeform) most master sources based on source templates
Thanks Tom!

#57 kbens0n

kbens0n

    Advanced Member

  • Members
  • PipPipPip
  • 3459 posts

Posted 07 September 2013 - 07:41 AM

The secret is to make a copy (for splitting) of part or all of the RM database by drag-and-drop to an empty database, or by doing a full copy of the database. Not via a gedcom export/import


In RootsMagic it is my understanding that drag-and-drop to an empty database -IS- a GEDCOM export/import (just hidden behind the scenes). Just an FYI.

---
--- "GENEALOGY, n. An account of one's descent from an ancestor who did not particularly care to trace his own." - Ambrose Bierce
--- "The trouble ain't what people don't know, it's what they know that ain't so." - Josh Billings
---Ô¿Ô---
K e V i N


#58 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6254 posts

Posted 07 September 2013 - 08:52 PM

Pheney's observation is that having used the ultimate splitting tool, the resulting sources based on the concomitant custom source template derived from the built in Obituary/Newspaper template do not transfer correctly via GEDCOM but do via drag'n'drop. He observed the same effect with other source templates which originally had a Source Details field at the beginning of the sentence template. Yet the split source looked fine in the originating and target databases via drag'n'drop which would suggest that there is a difference from GEDCOM. I cannot confirm with my own tests for a couple of weeks. If somebody else would test in the meantime it would expedite a conclusion whether there is or is not a bug with GEDCOM export/import.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#59 FOVeteran

FOVeteran

    Advanced Member

  • Members
  • PipPipPip
  • 43 posts

Posted 29 September 2013 - 12:31 PM

Thank you, Jerry, and others. I have been feeling somewhat guilty for my own at times extreme splitting and have ended up with a jumble of lumping, splitting and extreme splitting with not clear direction as to which was best. You have helped me to feel better about what is working best for me, but I would like to share about what is working for me--right now anyway. Perhaps someone will find my comments useful in this discussion.

First, I have moved away from the RM provided templates. They never seem to meet my needs and it takes too long to wade through them. I also gave up on the a couple of master templates idea, as it just didn't work for me and how I cite my sources. I spent a lot of time building a few complicated templates of my own, but some of them are so long that they don't all appear in the window and there is no scroll bar when I need it. In the end I find myself using over and over a simple free-form template I created myself. I just have a couple of templates I created I still use, but as Jerry mentioned, you can just copy another source and change it to create new sources, so the need for them is not as great as I had originally thought. Use only one or a few template sources reduces the amount of time spent hunting for the right template. All you have to have is a good understanding of the purpose and common syntax for citing to use a free form template of some kind.

Addressing the issues raised about difficulty creating good short footnotes and bibliographies when extreme splitting, I created additional fields in my Free Form Template which is pretty short:

Master Fields
Author (includes author details, such as email address)
Short footnote Author
Bibliography Author
Title (includes titles, sub-titles, type of source, clarifications, etc.)
Short footnote Title
Web host
Pub Info (includes publisher, place, date, etc.)
Short Pub Info (for short footnote)
Orig Pub info (where sources are republished, posted to the web, shared in email, etc.)
Source Note

Source Details:
Item Info
Short Item Info
Item URL
Date Accessed
Item Note
Short Item Note

Now I question if I should have any source details at all, as I have experienced the same problem Jerry did--where I decided to change the source details and had to change every entry where I used that sort to do so. I think it is going to depend on what type of source you have.

After this discussion, I'm feeling better about the advantages of extreme splitting. But, although I didn't read every post as I'm in a time crunch, I don't think anyone has addressed a couple of points.

One of the main reasons I lean toward extreme splitting is the ability to find a source in the source list and to identify the specific source in the individual's list of sources. For instance, it is very helpful when judging varying contradictory facts to know exactly where the information came from. So instead of a general source for say Obituaries....., I prefer to see "obit Doe, John 1865-1910". Or instead of "Arkansas Marriages...." to see "mrg rec Doe, John & Mary Smith 1899".

Note that I use a LOT of abbreviations--after all this is for my eyes only. That's because I want to be able to see more of the meat of the source in the viewing window. So instead of spelling out "Death certificate:" like Jerry did, I would just use lower case to denote type of source, abbreviate certificate to cert or record to rec or whatever, and leave off any punctuation whatsoever before the name - i.e. "death cert Doe, John 1910".

I don't know if my free-form template is going to move well into a gedcom, export, etc. yet, so if anyone sees a major problem with it off the top of your head, please let me know.

BTW, you can easily change from lumping to splitting a source by simply changing detail fields to master fields IF AND ONLY IF the detail fields are all the same at this time.

I wish I had seen this thread early on, but am glad I have read a little of it now.

#60 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6254 posts

Posted 30 September 2013 - 10:43 AM

@FOVeteran: I hesitate to concur that yours is a free-form template given that it has 16 fields making it as complex as the second-most complex built-in templates (the most fields is 17, iirc). Granted, it does have fields that allow for almost anything to be entered and fields for ShortFootnote and Bibliography that are variants of a corresponding Footnote field which makes them sentence specific. However, the former is true of some built-in source templates and the latter is accomplished without additional fields by the use of sentence template modifiers such as :Abbrev, :Surname, :Reverse. That said, I understand your goal is to have more control over the consistency of your citations than is delivered by the RM Free-Form 'template'.

For anyone to fully understand and comment on your template, we would need to see it in full. You can export it to a .rmst file which others can then import into a database to try out. Renee Zamora hosts a template-sharing website described at RootsMagic 4 User Created Source Templates, which is one way that you can share it.

Without viewing the Footnote sentence template, it's not possible to comment fully on the GEDCOM export. One thing I can say is that your use of Author, Title and Publisher fields won't result in the corresponding fields in other systems without a change to RM's export process - currently, all the parts of the Footnote sentence that come from Master Source fields along with embedded text end up in the GEDCOM TITL field. How that plays out from other systems can vary amongst them.

BTW, you can easily change from lumping to splitting a source by simply changing detail fields to master fields IF AND ONLY IF the detail fields are all the same at this time.

This I do not understand and would not expect nor does it work for me. You are saying that two identical citations of the same lumpy master source will become two citations of two identical master sources simply by moving all the source detail fields to the master source? I don't find that to be the case - there remains but one master source and some or all of the values of the former source detail fields are missing.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.