Jump to content


Photo

RM7 direct import from Family Tree Maker


  • This topic is locked This topic is locked
59 replies to this topic

#41 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3420 posts

Posted 17 March 2016 - 08:52 PM

Dragging and dropping the entire database should not - by default - lose or corrupt data.

 

I totally agree and have repeatedly affirmed my view that Drag&Drop should override the safeties so to speak.

 

I also must agree that as RM allows Description entries of >100 characters without warning then it should act to preserve this data at the very least in the Drag&Drop operation.

 

Lastly I am also disappointed that, from my memory, no official recognition or explanation of the present situation has been offered by RM.


We are all limited by our visions and abilities

Whilst we can borrow from the visions of others we cannot always deliver.

 

User of Family Historian 6.2.7, Rootsmagic 7.6.2, Family Tree Maker 2014 & Legacy 7.5

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#42 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 8470 posts

Posted 18 March 2016 - 08:17 AM

Here are some of the rules used for handling FTM descriptions during import. The problem is even on the same fact type it can vary on what type of information was entered into the description field. I would recommend that the user after the import go through the Place List. This will help them determine what records to clean by moving the place details back to descriptions. 

 

Occupation, Religion, etc it goes into the description. On birth, marriage, etc it goes into place detail.

 

It automatically enables description for all custom facts.

 

Facts with descriptions only, with no places entered will remain descriptions. If that fact doesn't have the description field enabled they still won't see it. They will need to enable the description field and add < [Desc]> to it.


Renee
RootsMagic

#43 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3420 posts

Posted 18 March 2016 - 12:45 PM

 I would recommend that the user after the import go through the Place List. This will help them determine what records to clean by moving the place details back to descriptions. 

 

Renee, in my opinion Rootsmagic really need to make the operation of switching/splitting fields much easier in the DataClean feature and elsewhere.

 

FTM had a very simple system of side arrows which would split out Place Details based on the comma delimiter and also a feature to move the data to the Description field if it were deemed to belong there. The current Rootsmagic functionality is very cumbersome to use and I do hope this is noted in the tracking system as an area where real productivity savings could be realized.

 

I have included a couple of mock up graphics below and such arrows could be extended to the Description field for much greater ease of moving these components about and a great time saver.

 

DC2.png

 

DC3.png


We are all limited by our visions and abilities

Whilst we can borrow from the visions of others we cannot always deliver.

 

User of Family Historian 6.2.7, Rootsmagic 7.6.2, Family Tree Maker 2014 & Legacy 7.5

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#44 mikelevoi

mikelevoi

    Member

  • Members
  • PipPip
  • 26 posts

Posted 18 March 2016 - 04:55 PM

There may be a difference between the direct and GEDCOM imports in the way they handle sources for the preferred name in the presence of multiple names ...

 

Tom,

 

As you are the SQLite expert par excellence - and RM obviously have no intent to fix the mess for FTM users who want all Descriptions in the Description field - regardless of whether a place exists or not - can I ask you a huge favour?

 

I have used your SQL to move all short NOTE to Descriptions. Can I ask you to modify the SQL to move all Place Details to Description please? This will never be a problem for FTM users as that field did not exist in FTM. I have looked at the code for moving NOTE to Description - but I really do not understand enough about database programming to do what I would like ;-(

 

If you write the code, I will package it for FTM user consumption ;-) The code I used for NOTE was found here:

 

http://sqlitetoolsfo... to Description

 

Many thanks,

 

Mike



#45 mikelevoi

mikelevoi

    Member

  • Members
  • PipPip
  • 26 posts

Posted 18 March 2016 - 05:03 PM

PS The solution from Renee to examine each entry and take appropriate action is not going to be possible. My database has 1400 people and 5000 events - other users have 30,000 or more people in them. It is not feasible to examine each entry manually ...



#46 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6254 posts

Posted 19 March 2016 - 09:31 AM

 

Tom,

 

... Can I ask you to modify the SQL to move all Place Details to Description please? 

I think that is fairly easy. Is it just the PlaceDetail.Name or also the PlaceDetail.Note? 

If both, is intervening punctuation, white space required?

Should there be a prepended heading, e.g., "***appended from Place Detail***", to facilitate search, inspection and replacement?

Should Place Details be deleted?

 

Edit:

Here is a snippet that produces a prototype NewDescription. It is just a query - no modification of database. Is that what you want the Description to be?

SELECT 
 E.Details 
 || CASE LENGTH(E.Details) WHEN 0 THEN '' ELSE '. ' END
 || '***appended from Place Details*** ' 
 || P.Name 
 || '. ' 
 || P.Note
AS NewDescription
FROM EventTable E
JOIN PlaceTable P 
ON E.SiteID = P.PlaceID
AND E.SiteID > 0
AND P.PlaceType = 2
;

Edited by TomH, 19 March 2016 - 09:47 AM.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#47 mikelevoi

mikelevoi

    Member

  • Members
  • PipPip
  • 26 posts

Posted 20 March 2016 - 03:56 PM

Quote

"I think that is fairly easy. Is it just the PlaceDetail.Name or also the PlaceDetail.Note?

If both, is intervening punctuation, white space required?

Should there be a prepended heading, e.g., "***appended from Place Detail***", to facilitate search, inspection and replacement?

Should Place Details be deleted?"

 

Thanks, Tom. Your query looks good to me.

 

- When I look at the database, in the table EventTable, the field called PlaceID contains a number.

- In PlaceTable, the record with that PlaceID has a field called "Name" which is the place name itself - like "London"

 

- In EventTable, there is also a field called SiteId which contains a number.

- In PlaceTable, the record with that PlaceID has a field called "Name" which is the place details aka description - like "He was born on Tuesday morning"

 

So, I would like to move the contents of the field Name from the record indexed by SiteID in PlaceTable to the Description in EventTable. To answer your questions specifically:

 

- Just Name

- Not both - Note will always be blank for FTM refugees

- Not prepended - this field did not exist in FTM

- Make Name ""/blank

 

I am happy to continue this discussion offline if you wish - so that we do not bore the rest of the forum ;-)

 

I am mikelevoi AT gmail DOT com

 

Mike



#48 mikelevoi

mikelevoi

    Member

  • Members
  • PipPip
  • 26 posts

Posted 21 March 2016 - 12:19 AM

Tom,

 

Your SQL works fine. So now I have all my Descriptions back where they belong ;-) I will package a solution for those FTM users who wish to go down this route.

 

Of course, if RootsMagician changes his mind and leaves Description alone - then the SQL code will not be needed. I must say that it is much nicer having the Description data in Place Details rather than in NOTE. As Place Details does not exist as a field in FTM, it makes it 100% perfect to move the Description data back. When it was moved to NOTE, there was no fool proof way to move it back as FTM users often used both fields.

 

Maybe RootsMagician should change FTM GEDCOM import to do the same thing? Just a suggestion to make FTM import via FTM or GEDCOM consistent.



#49 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3420 posts

Posted 21 March 2016 - 02:05 PM

I must say that it is much nicer having the Description data in Place Details rather than in NOTE. As Place Details does not exist as a field in FTM, it makes it 100% perfect to move the Description data back. When it was moved to NOTE, there was no fool proof way to move it back as FTM users often used both fields.

 

Maybe RootsMagician should change FTM GEDCOM import to do the same thing? Just a suggestion to make FTM import via FTM or GEDCOM consistent.

 

You just never know, someday ex FTM users may just decide to use the Place Details field for Place Details, then where will we be :D


We are all limited by our visions and abilities

Whilst we can borrow from the visions of others we cannot always deliver.

 

User of Family Historian 6.2.7, Rootsmagic 7.6.2, Family Tree Maker 2014 & Legacy 7.5

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#50 mikelevoi

mikelevoi

    Member

  • Members
  • PipPip
  • 26 posts

Posted 21 March 2016 - 05:50 PM

At the risk of harping on about this Description problem, I wonder if the RM7 developers have considered what on earth they are going to do when Ancestry rewrites the TreeSync API later in the year?

 

When TreeSync is available for RM7 use, how are RM going to support bi-directional data transfer when they have moved Description data to Place Details - and yet the data on Ancestry.com WILL be in the Description field?

 

Will they move the data only when place is specified? How is that going to enable seemless upload AND download of changes to and from Ancestry?

 

Is the data going to move when the Ancestry user adds a place in the online tree? Or move when a RM user deletes a place in the offline tree?

 

The possibilities for data loss appear to be endless ;-)



#51 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3420 posts

Posted 21 March 2016 - 06:08 PM

The possibilities for data loss appear to be endless ;-)

 

Ah, the bigger picture ;)


We are all limited by our visions and abilities

Whilst we can borrow from the visions of others we cannot always deliver.

 

User of Family Historian 6.2.7, Rootsmagic 7.6.2, Family Tree Maker 2014 & Legacy 7.5

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#52 KFN

KFN

    Advanced Member

  • Members
  • PipPipPip
  • 213 posts

Posted 22 March 2016 - 06:57 AM

I find all of this rather funny. If only the software both RM and FTM followed the GEDCOM standard AND taught their users what data belong in each field, then this would not be an issue. Case in point....

I find it interesting that "place detail" on the RM screen in GEDCOM terms becomes an ADDR tag, rather than a PLAC.NOTE tag. I would have thought without having been told that detail on a place was "verbiage" regarding the place rather than an address, since the ADDR tag is not a sub tag of PLAC but rather at the same level, _AND_ that the ADDR tag definition in GEDCOM states that this should be: "The address structure should be formed as it would appear on a mailing label using the ADDR and the CONT lines to form the address structure." Thus the ADDR tag is not an extension of the PLAC tag, rather it is (or could be) a redefinition of the data found in PLAC.

Allowing users to have only 3 data point (FTM) and 4 data points (RM) forces users to invent uses for this data entry points, only to find later that reports designed by the developer who thinks the user would enter one type of data in the field but in fact the user entered different data in each field. Each program should have extensive and immediately available written rules governing what data is appropriate for each data point and how it will be used throughout the program. I am accustom to reviewing a data definition document that defines data and its use. RM should publish one for its users and future converts.

#53 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3420 posts

Posted 22 March 2016 - 02:54 PM

I find all of this rather funny. If only the software both RM and FTM followed the GEDCOM standard AND taught their users what data belong in each field, then this would not be an issue. Case in point....

 

Computer genealogy in a nut shell, well stated.

 

That being said I do enjoy the level of quality that the Rootsmagic Place Details including Notes, Media and Geocoding bring to my database for eventual output in more detailed reports.


We are all limited by our visions and abilities

Whilst we can borrow from the visions of others we cannot always deliver.

 

User of Family Historian 6.2.7, Rootsmagic 7.6.2, Family Tree Maker 2014 & Legacy 7.5

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#54 mikelevoi

mikelevoi

    Member

  • Members
  • PipPip
  • 26 posts

Posted 23 March 2016 - 03:36 PM

Allowing users to have only 3 data point (FTM) and 4 data points (RM) forces users to invent uses for this data entry points, only to find later that reports designed by the developer who thinks the user would enter one type of data in the field but in fact the user entered different data in each field.

 

Well said - and here we have the crux of the problem. The users decide what to put in the fields - because there are so few fields to contain data - and the data they enter is manifestly not what the FTM or RM developer thinks they should use the field for ;-)

 

In light of the GEDCOM standard for descriptions being out of date and inadequate here - I would thought that the simple solution was for all developers to (a) simply not limit descriptions to stupid, arbitrary limits of 100 etc and/or (B) Do what RM have done and import other program data natively.

 

Having said this, RM can read the FTM database and yet they still move the data despite what the users want simply because they do not think another developer can handle their GEDCOM output if they are not "100% compliant". Guess what - the user simply does not care about this compliance ;-)

 

RM went through hoops to read FTM GEDCOM before they could read FTM direct. Does it work? Heck, yes. And this is despite the fact that FTM GEDCOM is probably the worst formed GEDCOM on the planet ;-) Even I can process FTM GEDCOM using VB.NET and do what I want with it to make it GEDCOM compliant - or not - as I see fit.

 

So - can we please have some sanity here and stop mucking about and adhering to some so called standard that users are not happy with?

 

PS I should point out that RM use of RESI is completely wrong according to the GEDCOM specification, as far as I can see. I quote:

 

INDIVIDUAL_ATTRIBUTE_STRUCTURE:=

[n

CAST <CASTE_NAME> {1:1} p.43

+1 <<INDIVIDUAL_EVENT_DETAIL>> {0:1}* p.34

...

n
RESI /* Resides at */ {1:1}

+1 <<INDIVIDUAL_EVENT_DETAIL>> {0:1}* p.34

 

So either be GEDCOM compliant - or not. It is wrong to do things to user data in the name of compliance when you do not comply.

 

- Why do you allow Description for RESI in exported GEDCOM?

if

- You export Description for DEAT as CAUS - when it is not a CAUSE - but simply a comment/Description - well, it is a comment the way I use the Description field ;-)

and

- You move Description to Place Details - when the fancy takes you.

 

Be consistent - or do what the user wants. Please do not do things using the excuse of being GEDCOM compliant when you are clearly not ...

Oh well - back to the real world ;-)



#55 KFN

KFN

    Advanced Member

  • Members
  • PipPipPip
  • 213 posts

Posted 23 March 2016 - 05:34 PM

You said: "In light of the GEDCOM standard for descriptions being out of date and inadequate here - I would thought that the simple solution was for all developers to (a) simply not limit descriptions to stupid, arbitrary limits of 100 etc and/or (B) Do what RM have done and import other program data natively."

Unfortunately your assessment here is wrong. But not due to anything other than not understanding the GEDCOM standard.

GEDCOM "descriptions" are NOT out of date, and probably not inadequate, in any way.


1) In GEDCOM a "FACT" has no such field called "description" so calling them or inferring that they are a description is initially wrong.

2) The location where FTM places the data from the "description" data entry field in the GEDCOM is incorrect. This location has specific defined data requirements, which if you read the documentation rarely exceeds or needs to exceed 100 characters

3) Only 3 of these defined data elements have a definition that approaches a "description"

a. DSCR tag known as "Physical Description" It is the only tag that allows for an unlimited value.

b. EDUC tag known as "scholastic achievement" 248 characters long. Defined as "A description of a scholastic or educational achievement or pursuit"

c. PROP tag know as "Possessions" 248 characters long, Defined as: "A list of possessions (real estate or other property) belonging to this individual."

4) Having looked at the data that may people place in the "description" field, I can see that GEDCOM has other tags (or fields) that this data can more appropriately be placed.

a. Names and ages of other people found in a Census should be placed in the "Notes" portion of the "Census Event" or better in the TEXT portion of the Source_Citation for that Event.

b. Additional documentation of a location of any event/fact such as: "The green house on the hill", or "they only lived here three months" (in a RESI fact) would be much better in the Place_Structure i.e. EVENT.PLAC.NOTE tag in the GEDCOM.

c. I could write an entire book on how to use the GEDCOM correctly. Yes it has its problems, no argument, but they are not what you have outlined here.

5) The DEAT tag has only one allowed value in the GEDCOM a 'Y' that indicates the individual is in fact "Dead". The 'Y' is only valid if and only if no other data is associated with the DEAT tag, such as a place or a date. If you have other information about a DEATh it probably has a place in the GEDCOM, if someone in the software world was smart enough to put it there.

6) If you have a DEAT and you know the "Cause of the Death" you should have a field available to enter "Cause" for that matter any event or fact that makes sense to have a "Cause" should have a data entry field for that cause.

7) If the person was married as a Catholic, Protestant, Jew or Muslim. This value does not go in a "description" field it goes in the RELI tag aka Religious_Affiliation a subtag of the MARRiage tag.

8) As noted above ADDR is a place to enter an address in the form of a mailing label. It is not appropriate to put a "description" or "place detail" here unless it is in the form of a multi-line address label. This would be great in a RESIdence tag.

9) Events and facts have a TYPE tag subordinate to the primary FACT/EVEN tag this tag is used to refine the superior tag. For instance A stillborn baby could have BIRT tag with a TYPE subtag of "stillborn" no need to put this data in a "Birth Description" field.

I hope this long introduction to GEDCOM use dispels the incorrect statement that the description is out of date or inadequate. And it also helps to dispell the notion that GEDCOM does not satify many of the needs of a genealogist. It is NOT GEDCOM, it is the programs and the programmers that don't know anything about GEDCOM and users who wrongly and unknowingly assume incorrect statements about GEDCOM that need education.

Again.... GEDCOM is not perfect, it has many shortcomings, but "description" is not one of them.

#56 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6254 posts

Posted 23 March 2016 - 06:07 PM

Given the many ways that GEDCOM has been understood or interpreted by so many different developers, I can only conclude that there is probably more than one that is arguably correct or consistent with the specifications. It is very difficult to write specs that preclude alternative interpretation. Then there is the motive for a software publisher to support a standard. Importing is almost invariably more important to him than is exporting, for commercial reason. I recall the DXF format for exchange among CAD systems, a field that should lend itself to greater precision for specs; data loss or misconstruing was a common problem between competing systems. It is a pipe dream that genealogy software publishers will ever converge to a common implementation of GEDCOM. Maybe there is a better chance with a successor but my sense of that is the software publishers are not heavily invested in one.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#57 KFN

KFN

    Advanced Member

  • Members
  • PipPipPip
  • 213 posts

Posted 23 March 2016 - 07:56 PM

Tom, I wish I did not agree that software publishers probably only care about getting new clients, while they block the loss of a client at ever turn, but I think your statement is right. Ultimately the data is mine, and I want it to be portable. I've used 4 different programs in about 30 years of genealogy, tried an additional 4. Most of them captured the same data, but all failed to transfer data to other programs well. The only reason we are here in this thread is because FTM almost went out of business and panic set in. RM has been forced to do some fast work to capture the hearts of individuals by fixing their import, and yet many FTM refugees are still not happy because FTM never did their homework, provided an inferior product and almost crushed its user base, all the while the users are blaming GEDCOM and/or RM.

What we as consumers of these programs need to do is remind the publishers that we want superior products, and require the products they produce allow us to take our work with us. If the software is good or great then I probably will never leave, if the software has flaws I'll ask that they are addressed, if other programs have some better points and some worse points maybe I'll own 2 or 3 programs using each one to their best ability, but transfering data between them. If not with GEDCOM then with direct read. The days of locking people into programs so they can't get at their data should be in the past. I know that at least with RM I can get at my data using SQL. Most other programs hijack the data so I can't get it out.

I have been thru dozens of database and application transfers in my life. All because one program did something better or different than the other. It pains me to see people lose work, because their is no good way to transfer data.

#58 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 8470 posts

Posted 24 March 2016 - 08:03 AM

I'm going to close this thread. If you have specific issues we need to address with the FTM import please open a new thread on that one topic.


Renee
RootsMagic

#59 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6254 posts

Posted 28 March 2016 - 02:26 PM

For those FTM emigres who object to certain FTM event descriptions being imported by RootsMagic into the event Place Detail, see FTM import - restore Event description from Place Details


Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#60 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 8470 posts

Posted 11 April 2016 - 08:14 AM

Closing this thread. Open new discussions as needed.


Renee
RootsMagic