Jump to content


Photo

gedcom export/import what will i lose?


  • Please log in to reply
12 replies to this topic

#1 maginnis

maginnis

    Advanced Member

  • Members
  • PipPipPip
  • 57 posts

Posted 17 March 2012 - 11:34 AM

last night my file got corrupted. i can open it fine but when i go to "people" tab, i get into a loop of errors. seems i had tried to add too many fields to this view.?? i'm able to export to gedcom. my question i hope is simple: what will i lose if i export and import my gedcom? i've heard some rumors, but i need to know what if anything will be lost. has anyone nailed this down?

#2 Romer

Romer

    Advanced Member

  • Members
  • PipPipPip
  • 2064 posts

Posted 18 March 2012 - 04:27 PM

What do you mean by a "loop of errors"?

Perhaps there's a way to recover from whatever issue you're experiencing that won't require a backup, but additional details as to what's occurring would be helpful. I'd think that if you had corruption in your database, a GEDCOM file might also be corrupted and/or incomplete.

#3 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6218 posts

Posted 18 March 2012 - 05:46 PM

GEDCOM and Drag & Drop will most certainly lose unused places, unused custom fact types, unused custom source templates, unused sources, unlinked media, ... And will leave behind orphaned records. None of that is necessarily a bad thing. What is of concern is losing anything that is used. On that front, we don't know for sure all of the losses. We do know that trailing white space is dropped from notes; if you have added carriage returns at the end of notes to force the following event to be a new paragraph, they will be gone. There was an issue with custom roles that may have been resolved by the last version of RM4.
You might recover from your problem with people view by deleting the file that stores program settings and custom views: RootsMagicUser.xml found likely in your User folder.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#4 Laura

Laura

    Advanced Member

  • Members
  • PipPipPip
  • 4276 posts

Posted 18 March 2012 - 07:04 PM

I was just checking, and I did not lose default role sentences or customized fact sentences when importing a gedcom into 5.0.2.1

I just customized 50 facts to the People view and got no error message, and when I scroll left all the facts are shown in the view. I picked the facts that I knew would be linked the most with most amounts of data entered and some were default and some user defined. So unless you added more than 50 facts to the People view and there is a limit, I doubt if that is the cause.

I don't show the customized facts for the People view in the .xml file. I think it is saved within the database file itself.

#5 maginnis

maginnis

    Advanced Member

  • Members
  • PipPipPip
  • 57 posts

Posted 18 March 2012 - 08:45 PM

the error is "SQLite Error 1 - at most 64 tables in a join. beneath that is another dialogue box to send report to rootsmagic. the loop is that each time i press OK, another one pops up until finally the program shuts down and then reboots again.

here's what happened: added a few facts to ppl view. did my work. removed some added some more. did more work. then i decided to add all the facts with "details". that is when the error started.

how is it again that rootsmagic looses data on it's own gedcom export/import? isn't that not a good idea, or am i missing something(asking seriously here)?

TomH, as far as anyone knows, it sounds like i don't have anything to worry about here unless it's an unused customization. i'll try deleting RootsMagicUser.xml.

#6 maginnis

maginnis

    Advanced Member

  • Members
  • PipPipPip
  • 57 posts

Posted 18 March 2012 - 09:27 PM

no joy after renaming that file. it actually looks like it gets written to every time the program opens. the loop i referred to happens 10times before spontaneous reboot. -maybe bruce can do something with that info. :) i have a fresh install of the program, so it must be the file itself.

hmmm, if i set the program to open in people tab and put a salt shaker on my enter key, i can burn in the new laptop.

seriously though, it's too bad. i lost 3hrs of work cleaning up places.

#7 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6218 posts

Posted 18 March 2012 - 10:15 PM

After Laura's post I went in search of some trace of People View settings and came up empty handed. Nothing in rootsmagicusers.xml and nothing in ConfigTable nor anywhere else I looked. Has to be somewhere! In the Windows Registry?!
Be that as it may, it sounds like the way RM builds its SQLite query for People View is to add a table join for each column added beyond the initial default. SQLite has had a 64 table join limit since 2007, raised from 32. The silly thing is that RM5 only has 23 tables so some are being repeatedly joined. That's very basic but flexible program design. I would like to think there is a better way but I can understand that programmers might never have anticipated a user wanting more than 50 or so columns - how do you usefully display them?
It would be interesting to try your database on another computer and see if the problem travels with it.
Otherwise, looks like GEDCOM or Drag & Drop to a new database is your best bet.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#8 maginnis

maginnis

    Advanced Member

  • Members
  • PipPipPip
  • 57 posts

Posted 18 March 2012 - 10:57 PM

After Laura's post I went in search of some trace of People View settings and came up empty handed. Nothing in rootsmagicusers.xml and nothing in ConfigTable nor anywhere else I looked. Has to be somewhere! In the Windows Registry?!
Be that as it may, it sounds like the way RM builds its SQLite query for People View is to add a table join for each column added beyond the initial default. SQLite has had a 64 table join limit since 2007, raised from 32. The silly thing is that RM5 only has 23 tables so some are being repeatedly joined. That's very basic but flexible program design. I would like to think there is a better way but I can understand that programmers might never have anticipated a user wanting more than 50 or so columns - how do you usefully display them?
It would be interesting to try your database on another computer and see if the problem travels with it.
Otherwise, looks like GEDCOM or Drag & Drop to a new database is your best bet.


i was cleaning up Place Details on all my Facts -for the most part moving them into Fact Notes. there are 48 Fact Place Details and i probably added 30 or so bec i got sick of viewing 4 or 5 at a time and then customizing the view again. i sorted by column to find where i had data. i think a dialogue box "oh, that's probably too many columns" would be appropriate as it happened as soon as i hit OK to add the columns.

#9 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6218 posts

Posted 19 March 2012 - 09:17 AM

After Laura's post I went in search of some trace of People View settings and came up empty handed. Nothing in rootsmagicusers.xml and nothing in ConfigTable nor anywhere else I looked.


Dug a little deeper into the ConfigTable and I found that the People View settings are stored there, in the binary field DataRec for RecID=1. Each column is defined by a set of XML fields, e.g.:

<PersViewCol11><FieldType>10000</FieldType><EventType>4</EventType><DataType>4</DataType></PersViewCol11>
for Column 11 (0 being the first column to the right of the person's name), where EventType 4 is the Burial fact, DataType 4 is the Place Detail for that fact.

It should be possible through some SQLite trickery to modify this setting thus reducing the number of columns (hence joined tables) below the 64 table limit and restore normal operation. Having said that, I proceeded to do so, deleting the defs for the last two columns in a sample database using SQLiteSpy. For the results to appear in RootsMagic, the database had to be closed and reopened; not to do so might result in something strange when doing further customisation of the People View.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#10 maginnis

maginnis

    Advanced Member

  • Members
  • PipPipPip
  • 57 posts

Posted 19 March 2012 - 09:44 AM

think i can give this a try. so, i should be able to leave a few columns and just strip out those i don't want. or would i leave the columns and mark the event type and data types to "0"?

#11 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6218 posts

Posted 19 March 2012 - 10:39 AM

think i can give this a try. so, i should be able to leave a few columns and just strip out those i don't want. or would i leave the columns and mark the event type and data types to "0"?


I don't know. The column definitions are numbered consecutively so I would think that it is not a good idea to delete anywhere other than from the highest down. And I would expect that merely setting values to 0 would not eliminate the attempt to add a JOIN to the query - the result may be another kind of error.
I will write up the procedure I followed and put it on the sqlitetoolsforrootsmagic wiki later today.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#12 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6218 posts

Posted 19 March 2012 - 01:50 PM

See People View Error - at most 64 tables in a Join for a SQLite procedure to get out of this problem without any effect on your data.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#13 maginnis

maginnis

    Advanced Member

  • Members
  • PipPipPip
  • 57 posts

Posted 19 March 2012 - 03:34 PM

Wow. Wow! Nice Job. thank you.