Jump to content


Photo

Merging duplicate facts (from a FTM 'refugee'

duplicate facts

  • Please log in to reply
14 replies to this topic

#1 Orijohnal

Orijohnal

    New Member

  • Members
  • Pip
  • 1 posts

Posted 11 December 2015 - 04:20 PM

I've just jumped ship from FTM and gone ahead and bought RM.  I'm exploring its features and have encountered only one real disappointment.   FTM was famous for creating duplicate facts within a person's record.  Fortunately, recent versions have had a tool for comparing and merging those facts without losing source informaton.  The tool also enabled the selection of the best fit location and description info.  All this was done starting from the equivalent of the RM's 'edit person' window.  This feature greatly eases the tedium of comparing, verifying and merging duplicate facts.

 

I've seen discussions of this topic for earlier versions of Roots Magic.  Is there a comparable feature that might give us FTM refugees some comfort?

 

 



#2 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3409 posts

Posted 11 December 2015 - 04:27 PM

RM supports the merging of people, but not the merging of facts. Rather, the merging of facts is a manual operation. Basically, you choose one of the duplicate facts to keep, edit it to suit your needs, and delete the other one.

 

The way RM's merging of people works, if there are duplicate facts between the two people, the merge keeps both facts and lets you make the decisions about how to proceed via manual editing as described in the first paragraph of this note.

 

Jerry



#3 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6147 posts

Posted 11 December 2015 - 10:54 PM

Automatic deletion of a truly duplicate event ought to be fairly easy to do with SQLite. Merging of nearly duplicate events is a bigger problem: http://sqlitetoolsfo...- Merge#MergingEvents

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#4 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3409 posts

Posted 11 December 2015 - 11:45 PM

I should have mentioned that even as a manual operation, it should be theoretically possible for a user interface to support true merging of duplicate facts, but where you are making the decisions about what to keep and what not to keep rather than an automatic process making the decisions. For example, what if a person has one birth fact for 1848 in Virginia and another birth fact for 12 Oct 1848 in Tennessee. Suppose your research indicates that the correct information is that the person was born 12 Oct 1848 in Virginia.  In theory, the user interface could present you with the various pieces of information all on one "merge fact" screen and allow you to indicate which information to keep. That way, you wouldn't have to delete the extra fact at the end. But RM doesn't work that way.

 

Also, remember that any fact merging really needs to clean up things like media and citations, not just the dates and places.

 

Jerry



#5 Allen Prunty

Allen Prunty

    Advanced Member

  • Members
  • PipPipPip
  • 94 posts

Posted 12 December 2015 - 04:32 AM

I would like to throw my hat in the ring on this one.  It would also be useful to merge duplicate places in with the "standardized" place.  I am getting tons and tons of duplicate places the more I use Family Search.  Being able to import/export directly into familysearch is great but it gets a lot of trash facts and information.

 

It would be great to be able to clean it up a bit.  Also the Roots Magic place cleaner will sometimes generate duplicate places.

 

Allen



#6 Ludlow Bay

Ludlow Bay

    Advanced Member

  • Members
  • PipPipPip
  • 867 posts

Posted 12 December 2015 - 10:29 AM

One can use Reports > Lists > Fact List: people with more than one of this fact type to list individuals with duplicate facts.   Of course, because the report is not interactive you must push the list to an isolated location (2nd monitor, print, etc.) in order to get to the edit window.   



#7 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 8316 posts

Posted 14 December 2015 - 01:46 PM

I would like to throw my hat in the ring on this one.  It would also be useful to merge duplicate places in with the "standardized" place.  I am getting tons and tons of duplicate places the more I use Family Search.  Being able to import/export directly into familysearch is great but it gets a lot of trash facts and information.

 

It would be great to be able to clean it up a bit.  Also the Roots Magic place cleaner will sometimes generate duplicate places.

 

Allen

 

There is a merge button on the Place List. I am assuming you mean you want it automated.


Renee
RootsMagic

#8 xkanuk

xkanuk

    Member

  • Members
  • PipPip
  • 6 posts

Posted 10 January 2016 - 02:49 AM

I too would like to throw my hat into the ring.  I'm a recent convert from FTM and am thoroughly enjoying all the webinars and rootsmagictv vids on offer.  

 

The crossover has provided me a great opportunity to prune my tree and one of the things that has been annoying me lately is the number of similar and mainly duplicate facts. For example I have a grandfather born in a township in a county in one source but only the county in another citation for the same source etc.  So I have basically 3 similar birth records that just need to be merged into one birth fact.  I reckon I can go in and massage the most generic fact but a merging of facts would be nice to have.



#9 zhangrau

zhangrau

    Advanced Member

  • Members
  • PipPipPip
  • 1474 posts

Posted 10 January 2016 - 06:01 AM

Collecting info from multiple sources will inevitably result in finding variations of facts - especially birth. My basic plan is to have a single Birth fact for each individual, showing my best understanding of the "correct" date and place. In the Birth Notes I place a sentence (or paragraph) explaining any variations and inconsistencies that I've found. The proposal for an automatic fact merge might be able to handle reducing from multiple to a single Birth fact, and properly combining any attached citations and media. But I find it hard to imagine the the automatic process would write the explanatory info for me.  I'll stick with manually merging facts, at least until I can afford a copy of IBM's Watson to help me....



#10 fitz

fitz

    Advanced Member

  • Members
  • PipPipPip
  • 121 posts

Posted 10 January 2016 - 06:39 AM

I also use what I believe to be the correct information for entry and then enter a source for each document I have and in the footnote abstract each document (which I always do anyway) showing differences in bold print.  All the information is there in footnote with variations clearly visible.  I might also put in explanatory sentence if variation is major.



#11 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3359 posts

Posted 10 January 2016 - 06:56 AM

I have to throw my hat in (again) on this growing problem as it's as old as the hills and getting to be a bigger problem if anything. Back in the days of RM3 and prior identical Places were automatically merged, the only thing we really had to worry about was duplicate events.and sources, now we have several contributors and the more we interact and sync with outside sources this can only become a greater problem, so what can RM deal with.

 

Places: One could easily say that identical Places in respect of Place/Std..Place/Abbre..Place should automatically merge but what about media and Notes which may be identical, complimentary or conflicting? I believe all Rootsmagic or any programmer could do, and should do, is write a utility which presents these seemingly duplicate entries side by side for user decision/interaction, that alone would be a step forward. A further step forward and a desirable feature in any quality program would be to employ the option to apply some fuzzy logic like Duplicate Search Merge to present Places side by side which MAY be duplicates.

 

Duplicate Search Merge: Presently whilst this very useful utility does not create duplication the process or merging does and every merged individual deserves examination and likely warrants editing at the time of merging. At present the way DSM works does not promote this, nor does it allow the dropping or further examination of events side by side. I have posted a wish for this some years ago and remain hopeful that the next incarnation of DSM more closely resembles the UI of File Compare. At present I merge in small batches and go back over each merge to clean up the information as can be seen here at time code 4:40 , pity I hadn't been so disciplined years ago.

 

Sources: These continue to be a problem further exacerbated by the introduction of Source Templates which Rootsmagic does not merge even when identical. The problem of Source Duplication can be seen demonstrated here and this also needs to be dealt with in the next version release.

 

Facts/Events: Well this has been around the longest and whilst Rootsmagic could and should be able to deal with identical duplicate facts they are invariably different in quality or attachments. The Print people with more than one of these facts is a broad brush with no concern as regards Dates so not always very useful except for the likes of Birth and Death dates. Whilst on the subject I have wished some time ago for the option Create Group under the Facts/Print options which would an easy addition and would help deal with Text Dates etc in a more productive manner. The script TomH linked to is very useful in creating a Named Group of all those individuals who have duplicate events of the same year so it will catch those events Jerry described with one fact date being 1848 and the second being 12 Oct 1848.

 

The management of duplicate information should be a key consideration within any prefessional database program. The failure to manage such duplication progressively reduces the effectiveness of the database, the searching and utilities which analyse the data leaving users bogged down in manually dealing with duplication rather than helping them progress their research, this is a recurring theme on these boards.


“Your most unhappy customers are your greatest source of learning.” -Bill Gates

 

The great Indian mathematician Aryabhat caclulated the value of pi at 3.1416, ~1500 years ago and without a computer!

 

 

User of Family Historian 6.2.7, Rootsmagic 7.5.9, Family Tree Maker 2014 & Legacy 7.5 (in order of preference)

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#12 bonsai123

bonsai123

    New Member

  • Members
  • Pip
  • 3 posts

Posted 16 January 2016 - 12:05 PM

Is there a way to copy and paste the sources under one fact or does it all have to be manually typed in.



#13 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6147 posts

Posted 16 January 2016 - 01:02 PM

When you have opened Citation Manager on a fact, highlight the source citation you want and then click on the Memorize button. Then when you have Citation Manager open on some other fact or some other person, click the Paste button to add the citation.


Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#14 dixieau

dixieau

    Member

  • Members
  • PipPip
  • 13 posts

Posted 12 July 2016 - 01:01 AM

As a convert from FTMM3 to RootsMagic 7 (both Mac and Windows), I agree with Vyger's posts.  

 

RM needs to update its user interface and navigation; ie when merging a duplicate person why not have tick boxes on the facts that you wish to merge, instead of taking all facts across?

 

I am working through a 7000 person tree that I merged and am left deleting a gazillion deplicated facts individually, I actually moved from Mac to Windows thinking this would be an easier process, but isn't, every step has to be mouse clicked on both platforms, why can't I use my delete key on keyboard to bring up a delete fact menu box and hit return?  Better yet, why can't we go into an edit view and select multiple facts to delete at once?

 

I love RootMagic BUT it is like stepping back in a time machine to DOS (sorry RM).

 

I found sqlitetoolsforrootsmagic and tried to install it on Windows 10 thinking it may cut my workload down, but alas I can't get it working...  Such a shame, almost every step I want to do requires multiple unnecessary clicks to get there and complete the task.  

 

I have to throw my hat in (again) on this growing problem as it's as old as the hills and getting to be a bigger problem if anything. Back in the days of RM3 and prior identical Places were automatically merged, the only thing we really had to worry about was duplicate events.and sources, now we have several contributors and the more we interact and sync with outside sources this can only become a greater problem, so what can RM deal with.

 

Places: One could easily say that identical Places in respect of Place/Std..Place/Abbre..Place should automatically merge but what about media and Notes which may be identical, complimentary or conflicting? I believe all Rootsmagic or any programmer could do, and should do, is write a utility which presents these seemingly duplicate entries side by side for user decision/interaction, that alone would be a step forward. A further step forward and a desirable feature in any quality program would be to employ the option to apply some fuzzy logic like Duplicate Search Merge to present Places side by side which MAY be duplicates.

 

Duplicate Search Merge: Presently whilst this very useful utility does not create duplication the process or merging does and every merged individual deserves examination and likely warrants editing at the time of merging. At present the way DSM works does not promote this, nor does it allow the dropping or further examination of events side by side. I have posted a wish for this some years ago and remain hopeful that the next incarnation of DSM more closely resembles the UI of File Compare. At present I merge in small batches and go back over each merge to clean up the information as can be seen here at time code 4:40 , pity I hadn't been so disciplined years ago.

 

Sources: These continue to be a problem further exacerbated by the introduction of Source Templates which Rootsmagic does not merge even when identical. The problem of Source Duplication can be seen demonstrated here and this also needs to be dealt with in the next version release.

 

Facts/Events: Well this has been around the longest and whilst Rootsmagic could and should be able to deal with identical duplicate facts they are invariably different in quality or attachments. The Print people with more than one of these facts is a broad brush with no concern as regards Dates so not always very useful except for the likes of Birth and Death dates. Whilst on the subject I have wished some time ago for the option Create Group under the Facts/Print options which would an easy addition and would help deal with Text Dates etc in a more productive manner. The script TomH linked to is very useful in creating a Named Group of all those individuals who have duplicate events of the same year so it will catch those events Jerry described with one fact date being 1848 and the second being 12 Oct 1848.

 

The management of duplicate information should be a key consideration within any prefessional database program. The failure to manage such duplication progressively reduces the effectiveness of the database, the searching and utilities which analyse the data leaving users bogged down in manually dealing with duplication rather than helping them progress their research, this is a recurring theme on these boards.



#15 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3359 posts

Posted 14 July 2016 - 02:11 PM

I've just jumped ship from FTM and gone ahead and bought RM.  I'm exploring its features and have encountered only one real disappointment.   FTM was famous for creating duplicate facts within a person's record.  Fortunately, recent versions have had a tool for comparing and merging those facts without losing source informaton.  The tool also enabled the selection of the best fit location and description info.  All this was done starting from the equivalent of the RM's 'edit person' window.  This feature greatly eases the tedium of comparing, verifying and merging duplicate facts.

 

I've seen discussions of this topic for earlier versions of Roots Magic.  Is there a comparable feature that might give us FTM refugees some comfort?

 

 

 

A timely reminder of the counter productive problem that duplication is and with online and person to person interaction it is becoming almost viral in destroying the quality of genealogy databases.

 

For Sources, Facts, Places and Individuals I believe utilities must be written to facilitate the finding and merging of all these areas of duplication. One user records the full State name, one the abbreviation, one the Country the other not, duplication upon duplication and ever lessening file quality. In a sort of evolution of the File Compare UI once such possible duplicates are presented the user should be presented with all options necessary to merge.

 

With regards to Places I have said more than once that the Standardized Place Name should be the point of focus for duplicate identification and hopefully it will be in the future. Searching for possible duplicates where one user records the place country and another does not, one the state name and another abbreviation can be effectively dealt with by searching the Place name only to a certain depth defined by the user.

 

Orijohnal describes how more recent versions of FTM provide a facility to merge duplicate facts, another competitor provides various other partial match searching which greatly helps in finding duplicates, see below.

 

dsm-extra-merge-options.png


“Your most unhappy customers are your greatest source of learning.” -Bill Gates

 

The great Indian mathematician Aryabhat caclulated the value of pi at 3.1416, ~1500 years ago and without a computer!

 

 

User of Family Historian 6.2.7, Rootsmagic 7.5.9, Family Tree Maker 2014 & Legacy 7.5 (in order of preference)

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root






Also tagged with one or more of these keywords: duplicate facts