22 replies to this topic

[NEreswearcher]

I have been trying to export RM data to use in another application and I have been encountering an error that is confusing to me.

 

If I export a project with standard fact types without RM extra details I get the following in the GEDCOM file:

 

Baptism fact without alterations.

 

Baptism, Date, Place, PlaceDetails, Note

 

RM generated this code in the GEDCOM FILE:

1 BAPM

2 DATE 1 JAN 1900

2 PLAC Kalamazoo, Michigan

2 ADDR 50 Main St.

 

If I modify the Baptism fact type to include the description field as such.

 

Baptism, Date, Place, PlaceDetails, Description, Note

 

RM generated this code in the GEDCOM FILE:

1 BAPM Baptism Description

2 DATE 1 JAN 1900

2 PLAC Kalamazoo, Michigan

2 ADDR 50 Main St.

 

The same type output is generated for my custom facts:

 

Criminal, Date, Place, PlaceDetails, Description, Note

1 EVEN incarcerated for burglary for 10 years

2 TYPE Criminal

2 DATE 1 JAN 1880

2 PLAC Kalamazoo, Michigan

2 ADDR Michigan State University

2 NOTE Criminal Note

 

Criminal, Date, Place, PlaceDetails, Note

1 EVEN

2 TYPE Criminal

2 DATE 1 JAN 1880

2 PLAC Kalamazoo, Michigan

2 ADDR Michigan State University

 

A Familyhistorian V6 import gives me an error on the first criminal only and the second BAPM only :

l.53 - INFO ONLY: Detected & fixed invalid use of EVEN (event) tag: "1  EVEN Criminal Description"

 

Microsoft Validator gives me an error on the first criminal only and the second BAPM only:

R401      65   it is recommended to export structured addresses only.

 

When I import into another software package (LegacyFamilyTree) I get import errors saying the event tag in this case BAPM or Criminal cannot have a value on the same line as the BAPM/Criminal fact type.

 

All this is dependent on the GEDCOM Standard. 5.5 is relatively the same as 5.5.1 in this area.

 

Examples in the GEDCOM 5.5 standard.

 

1 EVEN

2 TYPE Awarded BSA Eagle Rank

2 DATE 1980

 

gedcom_line :=

level + delim + [xref_id + delim +] tag + [delim + line_value +] terminator

level + delim + optional_xref_id + tag + delim + optional_line_value + terminator

for example:

1 NAME Will /Rogers/

 

It appears that the line value is optional and that seems to be supported by their examples.

When I look at this I wonder if

1. you can’t have a value on the Even line or
2. you can have a value on the Even line or
3. you should accept both?

It appears that it there is 3 to 1 for not having a value on the even line so which approach is correct?

[KFN]

Okay, so here goes from a GEDCOM v 5.5.1 standpoint which is the most recent and almost best version!!

 

 

RM generated this code in the GEDCOM FILE:

1 BAPM

2 DATE 1 JAN 1900

2 PLAC Kalamazoo, Michigan

2 ADDR 50 Main St.

 

Your first entry is GEDCOM v 5.5.1 correct.

 

 

RM generated this code in the GEDCOM FILE:

1 BAPM Baptism Description

2 DATE 1 JAN 1900

2 PLAC Kalamazoo, Michigan

2 ADDR 50 Main St.

Your Second entry is GEDCOM 5.5.1 incorrect.  

Why?  Because BAPM is an "Event Type" where all tags of this type (except BIRT and DEAT) disallow a value following the tag itself.  This is a common practice by most programs that don't support the entire GEDCOM specification because they have no place to put this data in the GEDCOM structure.

 

 

1 EVEN incarcerated for burglary for 10 years

2 TYPE Criminal

2 DATE 1 JAN 1880

2 PLAC Kalamazoo, Michigan

2 ADDR Michigan State University

2 NOTE Criminal Note

Your third entry is GEDCOM v5.5.1 invalid. 

Why?  Because technically as stated above "Event Type" tags do not allow anything to follow the tag itself.

But.... Many programs see this as an error in the design of GEDCOM v5.5.1 and either allow information here (for EVEN tags only) because it would make it similar to the FACT tag and allow for users of the EVEN tag to actually create a tag with a sub classification value.

 

To understand this last statement I submit the following for your understand:

 

A standard Birth record would look something like this.

1 BIRT
2 DATE 02 OCT 1822
2 PLAC Weston, Madison, Connecticut

 

But you could enter the following:

1 BIRT

2 TYPE still born
2 DATE 02 OCT 1822
2 PLAC Weston, Madison, Connecticut

 

Many time a family will name a child that was still born and actually record its birth date in the family bible.  The above "TYPE" tag would allow the GEDCOM file to indicate that the child was still born and is a "sub classification" to the primary tag.  This is allowed for many tags, MARR for example could have a type of 'civil' or 'religious' or 'not married' for example.

 

The EVEN tag (unlike the FACT tag) does not allow for a sub classification.  I do this all the time however with GEDCOM:

 

1 EVEN Induction

2 TYPE Military

 

1 EVEN Discharged

2 TYPE Military

 

Conclusion:

Technically the GEDCOM v5.5.1 does not allow anything following an 'Event Type' tag (Most 'Attribute Type' tags can have a value following the tag).  EVEN is no exception, BUT... Many see this as an error in the document and have corrected that in their EVEN tag use.  Others have gone farther and allow all 'Event Type' tags to have data following the tag.

 

Side Note:  I don't think RM7 allows for a TYPE (sub classification) following any tags.  Therefore if following this concept to its natural conclusion, the EVEN tag should NOT have a value following the tag because 'sub classifications' are not allowed.

[KFN]

I just reread your entry and saw this message from Microsoft:

 

R401      65   it is recommended to export structured addresses only.

I think in GEDCOM 5.5.1 this is an incorrect statement.  However in v5.4 and with 5.5 this would be correct.

 

The Address_Structure added ADDR1, ADDR2 and CITY tags to the single line for address in previous release.

 

BUT in v5.5.1 GEDCOM says the following:

 

The address structure should be formed as it would appear on a mailing label using the ADDR and the CONT lines to form the address structure. The ADDR and CONT lines are required for any address. The additional subordinate address tags such as STAE and CTRY are provided to be used by systems that have structured their addresses for indexing and sorting. For backward compatibility these lines are not to be used in lieu of the required ADDR.and CONT line structure.

Which tells we that you should not use anything but the the ADDR and CONT tags to enter an address and that they are required.

 

I think in some programs don't like this because they have individual fields for city and state.  But this makes no sense to me if you are entering international addresses because many places have other address designations that are not easily shoe horned into this structure.  

 

I don't do much with this structure myself since I put street address information in my PLACe tag and dont care about 'mailing labels' and mailing information.

[NEreswearcher]

KFN

 

If I read your answer correctly then RM7  is generating invalid GEDCOM statements. In my example the data following the EVEN tag generated by RM 7 is from the description field and is rightfully rejected by other applications. I guess I will have to wait until RM corrects this. So far I have not found a way to correct this on my own. Some have suggested changing all EVEN tags to ATTR but so far that has not worked smoothly for me.

 

Should I address this to RM as an issue or error?

[KFN]

Should I address this to RM as an issue or error?

You can try, but I doubt it will change.  
 

I think RM gives you the ability to turn off the “description” field for all ‘Event type’ tags/facts.  This way you would not violate GEDCOM.  It is your choice to use the description field or not.  Using it causes the error in the GEDCOM.

The solution is, don’t use the ‘Description Field’ for ‘Event type’ tags.

[TomH]

You can address this to RM in whatever way you want.

If I understand KFN's explanation, while a value following the EVEN tag is incorrect in 5.5.1, it is regarded by many, including himself, as too restrictive. I am curious to know what FH and LFT do with that value which they have flagged as incorrect: discard or preserve (where?).

You can force RM to exclude the Description field value from being appended to EVEN which would then 'correct' that export 'error' but now you've lost the field from the user interface and the value from the GED file. It is still in the database but you would want to move it to the Note field (where else could it go?).

[KFN]

 

Tom said: “but you would want to move it to the Note field (where else could it go?).“

I agree with Tom on this.

 

To take the Event a little more down the road, I would create a different custom event.

 

Criminal, Date, Place, PlaceDetails, Description, Note

1 EVEN incarcerated for burglary for 10 years

2 TYPE Criminal

2 DATE 1 JAN 1880

2 PLAC Kalamazoo, Michigan

2 ADDR Michigan State University

2 NOTE Criminal Note

Would be changed to:

1 EVEN 

2 TYPE Incarceration 

2 DATE FROM 1 JAN 1880 TO 1890

2 PLAC Kalamazoo, Michigan

2 ADDR Michigan State University

2 NOTE Incarcerated for Burglay, 10 years.

 

Exact dates can be entered or just years if unknown.

 

If his profession was ‘Criminal’.

 

Then you could add an OCCUpation tag as well.

 

1 OCCU Criminal

...

 

Using the Description field with this tag is fine, since it describes his occupation.

[NEreswearcher]

TomH

 

FH(6) does a couple of things.

First it moves the data following the EVEN tag and move it to a field that they call "label" and it is addressable in their sentence structure.

Second, the bad news here is they don't recognize the GEDCOM  and therefor do not try to match the fact tag that they have with the one coming in and they create, as best that I can determine, a unique event that does not tie into a fact and takes a generic sentence structure.

FH does not lose data but loses the connection between the incoming fact and a fact type that already exists in FH

 

LFT(8)  recognizes the description field and that looks ok but somehow the note for that fact appears to have been lost. I was pleased when I looked at LFT they have done a much better job of importing than FH. They even created fact types for my new facts minus the sentence structure because I did not export the "extra" information from RM. I would need to create fact type in LFT just like I would with FH

 

FTM takes a long time to load on my machine so I did not retest that interface.

 

KFN

 

If I understand your suggestion I would get rid of the criminal fact type, I would create a new fact type called incarceration with no description field and then add another event for his occupation, as a criminal. Sounds like that would work,

 

I will try that and see how that works.

[KFN]

I’ll adjust terms a little here to be precise as I can as they pertain to GEDCOM. RM calls everything a fact, which is ok for them within their interface, but in GEDCOM they have different meanings.

 

I suggest creating a custom event (RM calls everything a ‘fact’) resulting in an EVEN tag with a TYPE of ‘incarceration’.  I’m doing this because ‘criminal’ is not an ‘event’ it is an ‘Attribute’, while ‘incarceration’ is an event in the person’s life. Since an occupation is an attribute of a person it can be denoted by the tag OCCU. I would say that this is probably this person’s occupation for some period of time and enter this in your database.

[KFN]

 

 

FH does not lose data but loses the connection between the incoming fact and a fact type that already exists in FH

I’m not sure what this means, however, one of things I rail on and on about is how so many genealogy programs just don’t play nice with each other and in some cases don’t use or understand GEDCOM.  

 

IF FH has a ‘fact’ called “criminal” but does not except an incoming EVEN tag properly setup for the custom tag of criminal then that is a bug in their import.  
 

My question would be, if you enter in FH criminal as a fact, the what do they output in a GEDCOM?  They could create some other type of GEDCOM that you may have to try to generate out of RM.

 

This would take more investigation.  These inconsistencies between programs is very frustration, and one of the reasons I can’t use multiple programs in genealogy, but I’d be more than happy to purchase 3 or 4 different ones if they did not lose, misinterpret or other wise screw with my data.  Some programs have better reports, others have better interfaces, Some store more data points, some are more open, while other ones live only on the internet and allow family to edit and review in real time.  Since they don’t play nice I pick the one that does the best I can expect.

[TomH]

The conversion of RM events having values in the Description field disallowed under 5.5.1 to having those values in their Note is easily done with a SQLite statement applied to the database.

A conversion that would see the creation of additional facts such as Occupation is possible using SQLite but more complicated.

[Jerry Bryan]

I have read this thread with great interest. I have had the (apparently incorrect) sense for a long time that the Description field transferred pretty well. So I have used it as sort of a short note with a number of fact types. I very much like that the Description field is supported by People View. Making the Description field into a column in People View and filtering People View by an appropriately defined Group is a very powerful way to work. It can make very visible minor data entry differences for the same kind of information for different people. I really wish People View supported the Note field. I think the stated reason why it doesn't (too much data) is extremely weak.

 

Another way I have used the Description field is to have two versions of the same fairly short note - one version of the note in the Description field and the other version of the note as a {private note} in the actual Note field. So in RM reports, I can print the Description field and not the Note field. And then I can share the same data with GedSite to make a Web site where I can not display the Description field and instead display the Note field which I un-privatize in GEDCOM export. The data in the privatized note in RM becomes a URL to a Google Map with a pin to a grave site in Gedsite. GedSite does not complain about RM's GEDCOM.

 

I think I am going to continue with my current practice. If ever I wanted to switch from RM to one of its competitors that won't import the Description field, I would use SQLite to move the Description field to the front of the Note field before converting my RM data to the RM competitor.

 

Jerry

 

[NEreswearcher]

Jerry,

 

I like RM, the way it handles data input was one of the major reasons that I went to RM from TMG. I don't want to reinvent the wheel so if I can't come up with a pretty seamless way to get from one product to another I probably wont. So far the one exception is FH since I can export and import data and generate charts without any problem so I will probably continue with that. They do have some capability that no one else has.

 

I think my original question has been answered. All the other discussion here are very interesting and important going forward but I don't foresee any immediate alterations in any software to facilitate sharing or moving data to accommodate a user using the various features of several software solutions to get their job done. I currently own 7 genealogical programs, three from RM, and the cost is relatively minor so I think more licenses would be sold rather than fewer if data exchanged easily and without error.

 

Tom I have used SQLite on several occasions to correct situations with the RM data base. I use it quite often in PersonalHistorian to do several functions that should be part of the product but I don't think I want to use sqlite to fix this issue until I really know what I want to do. I could dig an even deeper hole.

 

KFN

 

What i am referring to is FH imports all the data and if I get around the description field being on the same line as the fact type then FH matches the incoming fact type with the one in their system and uses all their sentence structures, etc., just as you would hope. If they see an error in the EVEN tag then they do not try to match fact types and generate a one of a kind fact with a single canned sentence and when looking at their fact types after import the new fact type is not there. works but not correctly in my mind.

 

I understand what you are saying about an EVEN with a type of criminal but in RM I'm not sure how to do that. I also don't know how to create an ATTRIBUTE that would satisfy many situations. An EVEN tag would be an actual event while the attribute would be a more permanent long lasting characteristic like he was five feet tall. I think there will need to be more clarification on GEDCOM so everyone see it the same. There also should be extensive development to meet our needs (actually mine) but again I am not holding out hope on that issue.

 

I'm like you in being very frustrated with some of this. While I think the new interface in RM8 will be nice I think fixing some of these issues would be more desirable and helpful to me.

[TomH]

I believe that you simply create a custom fact type named "Criminal" with the Description field disabled or left blank in all instances. It will export from RM as per 5.5.1 as a valueless EVEN of TYPE Criminal. With Date field enabled and populated, it is a time-based event. If disabled or left blank, it is still an EVEN on export but is equivalent to an ATTRibute.

[NEreswearcher]

Tom,

 

I will try that and see how FH treats it. It seems that RM will not create an actual ATTR GEDCOM export.

 

Thanks AGAIN TO ALL.

[NEreswearcher]

I have an additional question on managing fact types.

 

I've reduced all my custom fact types to the bare minimum.  I've created a test database where all those new fact types reside. I can create a new project and import a list with the fact types from this test database. All this works fine but if I want to change or upgrade a sentence it appears that I cannot update the test database and then re-import into my project. It appears that this process will only add new ones but will not update a current fact type thus making my maintenance one of updating a sentence wherever it exists.

 

Is there a better way than this?

[TomH]

An alternative way not provided by RM is to use a SQLite query to update the FactTypeTable in the target database from that of the reference database. That can work reliably for the built-in fact types because the primary key, FactTypeID, is consistent among all RM4-7 databases. Custom fact types that are intended to be identical among databases can have differing primary keys so one would have to rely on the fact name or abbreviation to be unique and consistent as a key to match them up.

[NEreswearcher]

Thanks Tom,

 

I was thinking that RM did not have a way to do this. I have used SQLite on many occasions and it fills in gaps the RM has left us with.

 

Sentences are stored in at least two tables. The FactTypeTable for the primary sentence and in the RoleTable for all the sentences associated with roles.

 

I think on an initial creation of a new project I can import lists to get all my new fact types however if I have modified a built in fact type they will not get updated by RM so I will need to keep a list of all fact types and if modified or not and utilize SQLite for any updates to sentence structures including if I modified a sentence on those built in facts.

 

Thanks Tom I was hoping someone had found an RM feature that I had missed.

 

p.s. I will need to review "BLOB" type fields to see if I need to do anything different with those.

[TomH]

p.s. I will need to review "BLOB" type fields to see if I need to do anything different with those.

 

I've seen BLOB fields flipped to TEXT with no apparent adverse effect (provided the contents were text anyway, e.g., sentence templates or Notes). Unsure whether all such cases were due to my manipulations with SQLite or to some operation in RM itself because I've seen examples that I didn't think I had touched. Even a mixture of the two types in the same column of a table. Just be aware that any text or string operation on a BLOB, e.g., TRIM(), automatically converts the result to TEXT.

[Jerry Bryan]

Sentences are stored in at least two tables. The FactTypeTable for the primary sentence and in the RoleTable for all the sentences associated with roles.

 

Sentences are stored in tree tables in RM.

1. FactTypeTable for sentences for facts when the sentences have not been customized for a specific instance of a fact.
2. EventTable for sentences for facts when the sentences have been customized for a specific instance of a fact.
3. RoleTable for sentences for roles. Unlike the sentences for facts, the sentences for roles cannot be customized for a specific instance of the role.

One thing that I used to do myself and also that I occasionally recommended that others do was item #2 in my list. In particular, if you are having difficulty getting a particular fact sentence for a particular person to read the way you want it, you can always just customize it anyway you want. And you can customize it with fixed text instead of with clever use of variables. But I finally concluded that doing so was a really bad idea because it essentially was storing data where metadata was being stored.

 

I view sentences as metadata rather than as data. For example, if ever I was to convert my RM database to some other system and if the other system wouldn't import my RM sentences, no data would really be lost. I probably could set my sentences up correctly in the new system in just a few hours. By data, I mean names, dates, places, notes, and relationships. I really think the sentences are only metadata. They are not metadata in the same sense the a database schema is metadata, but I think sentences nevertheless are genealogical metadata.

 

I have never been a TMG user but I did purchase a TMG license after it went defunct. At that time I came into contact a great deal with many TMG users to help answer questions about how to do TMG type stuff as RM type stuff. As a result, I have learned a great deal about TMG without ever done anything with it myself except to play around with it and to import the TMG sample database into RM to see how TMG data imports into RM. I'm very impressed with TMG and it's easy to see why the TMG users liked it so much. But if I had one observation about TMG and the way it tends to be used, it would be that the TMG data and the TMG sentences are too tightly bound together. TMG data and TMG sentences can be used together to produce very nice narrative reports. But the tight binding of the data and the sentences seems to me to lock you too much into the TMG world.

 

Partly in response to learning more about the TMG world, I have chosen to got the opposite direction with RM. I used to do a lot of customization to make my RM narrative reports read as nicely as possible. I didn't have my RM data and my RM sentences quite as tightly bound up together as seems to happen with TMG data and TMG sentences. But it was close. When I finally went the other direction and tried to disentangle my RM data from my RM sentences, I decided to go with point form sentences and more notes. A big thing I had to change was that my Census sentences were highly customized on a case be case basis. So I had to move that customization into the notes for my Census facts. It was a big job, but I think it was worth it.

 

I am now heavily invested in publishing my RM data as HTML pages using a tool called GedSite. GedSite is a successor product to SecondSite and is very much a part of the TMG world. Except that GedSite works with GEDCOM and can therefore publish data as HTML pages from any genealogy software that can produce GEDCOM. GedSite works very hard to import GEDCOM completely including customization in the GEDCOM put there by software such as RM. So it will import RM's sentences. But I quickly came to realize that the sentence formats I'm using in RM for printed reports don't work quite as well on HTML pages, so I define all my GedSite sentences in GedSite rather than using the RM sentences. I think this illustrates the advantages of separating the genealogical data of names, dates, places, notes, and relationships from the genealogical data of sentences.

 

Jerry