Jump to content


Photo

Duplicate Search Merge Improvements & Opinions

DSM Duplicates merge tools

  • Please log in to reply
12 replies to this topic

#1 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3183 posts

Posted 27 October 2013 - 02:51 PM

I would like to revisit the shortcomings of Rootsmagics Duplicate Search Merge when measured against the flexibilities of Legacy Family Tree.

I am no fan of the look of Legacy but their duplicate merge utility allows comparison of most important information on one screen through the use of tabs. Legacy also facilitates the copying of events from Duplicate to Primary person and the dropping (deleting) of duplicate events or just less accurate event information.

At present, in RM, I always select Merge and then worry about editing afterwards as my fear is accidentally dropping important information. This leads to a build up of unnecessary duplicate information which then needs to be dealt with at a later date.

The ability to make these changes on a side by side UI is much more useful than the present RM UI and RM could easily leap ahead by simply adding more information and options to the existing colourful screen like the mockup below. I do hope some improvements are made to this ever more important utility in the next version release.

Posted Image

If one accepts second best then quite often that's just what one will receive

— Dag Hammarskjold

 

Current user of Rootsmagic version 7.5.7.0, Family Tree Maker 2014 and Legacy 7.5 on Win 10

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#2 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3060 posts

Posted 27 October 2013 - 08:02 PM

At present, in RM, I always select Merge and then worry about editing afterwards as my fear is accidentally dropping important information. This leads to a build up of unnecessary duplicate information which then needs to be dealt with at a later date.


The part of this process that bothers me the most is the "worry about editing afterwards" as my fear is that I will never find the merged people again if I let any of the various automatic merges take place. Hence, I never use any of the automatic merges. Instead, I do completely manual merges - one person at a time - and completely clean up that person before going to the next person to merge manually. It's not a very efficient process, but at least it's pretty safe and it doesn't build up a huge backlog of individuals with information that needs to be dealt with at a later date.

Also, the automatic merges run the danger of merging people that shouldn't be merged. For example, I have in my database several sets of unnamed twins (and one set of unnamed triplets) who died at birth. Obviously, they have the same birth date and can look awfully mergeable to an automatic process that doesn't really understand what is going on.

So it goes without saying that I agree that the whole merging process in RM is in need of some pretty major improvements.

Jerry

#3 Ludlow Bay

Ludlow Bay

    Advanced Member

  • Members
  • PipPipPip
  • 861 posts

Posted 27 October 2013 - 08:31 PM

Also, the automatic merges run the danger of merging people that shouldn't be merged. For example, I have in my database several sets of unnamed twins (and one set of unnamed triplets) who died at birth. Obviously, they have the same birth date and can look awfully mergeable to an automatic process that doesn't really understand what is going on.


Would be nice to have a "do not merge" flag that could be set for any two individuals, to be referenced in subsequent merge operations - similar to the "not a problem" flag in the problem list.

#4 zhangrau

zhangrau

    Advanced Member

  • Members
  • PipPipPip
  • 1356 posts

Posted 28 October 2013 - 03:42 AM

For facts such as birth or death, the process keeps both the Primary and Duplicate events - merging them only if the date and place are identical, and there are no notes attached to either. If there are notes attached to either Primary or Duplicate, then the process keeps both events. But at least it preserves all of the Sources and related notes.

Manual or automatic merge, one of the problems I've noted before is that Alternate Names are very poorly handled.

For Alternate Name facts (yes, they ARE on the Fact List) the situation is woefully insufficient. There are several different outcomes available when merging Alternate Name facts, none of which match the behavior of the other facts. Even if the Primary and Alternate records include the exact same Alternate Name, the sources for the Duplicate are NOT merged into the Primary. In addition, it is common for the Duplicate Alternate Name fact to be copied into the Primary record, but WITH NO SOURCE. The only way to maintain documentation of the various Alternate Names for an individual is to MANUALLY copy the Alternate Name(s)and Source(s) into the Primary. The RM software really isn't helping at all.

#5 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3183 posts

Posted 28 October 2013 - 04:48 AM

So it goes without saying that I agree that the whole merging process in RM is in need of some pretty major improvements.


I believe the Rootsmagician is not a fan of clutter but I believe more information needs to be presented on this screen to enable a more informed choice and that nneeds to be in a side by side comparison display.

Would be nice to have a "do not merge" flag that could be set for any two individuals, to be referenced in subsequent merge operations - similar to the "not a problem" flag in the problem list.


Agree

For facts such as birth or death, the process keeps both the Primary and Duplicate events - merging them only if the date and place are identical, and there are no notes attached to either. If there are notes attached to either Primary or Duplicate, then the process keeps both events. But at least it preserves all of the Sources and related notes.


At least for the Birth Fact Legacy has checkboxes for keep left, keep right, keep both, also when an event is highlighted, like the Baptism event in my mock up above, there is the facility to merge these events on that screen, something which has been wished for before. I have not tested that with Notes, Sources, Media etc and at present don't know how it handles conflicting dates but will endeavour to test that tonight.

DSM is a powerful utility, it's just a pity that it is restricted in its flexibility at the important decision making step.

If one accepts second best then quite often that's just what one will receive

— Dag Hammarskjold

 

Current user of Rootsmagic version 7.5.7.0, Family Tree Maker 2014 and Legacy 7.5 on Win 10

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#6 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 7919 posts

Posted 28 October 2013 - 10:06 AM

Confirming enhancement requests are in our tracking system.
Renee
RootsMagic

#7 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3183 posts

Posted 28 October 2013 - 10:28 AM

.........there is the facility to merge these events on that screen, something which has been wished for before. I have not tested that with Notes, Sources, Media etc and at present don't know how it handles conflicting dates but will endeavour to test that tonight.


Just to confirm there is NOT an option in Legacy to merge events on that screen, my mistake. :rolleyes:

If one accepts second best then quite often that's just what one will receive

— Dag Hammarskjold

 

Current user of Rootsmagic version 7.5.7.0, Family Tree Maker 2014 and Legacy 7.5 on Win 10

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#8 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3183 posts

Posted 08 November 2015 - 12:19 PM

I have just uploaded a short screencast which will hopefully help some users who have expressed som confusion over Duplicate Search Merge in the past on these forums.

 

My main reason for the production was to provide a visual demonstration of the difficulties of widening the scope of DSM especially when working with larger databases. I believe if Bruce can facilitate restricting the scope of DSM to within the confines of a User Defined Group this would be a great help to those dealing with dupliccation in the future.

 

https://youtu.be/T8JH-3Tf3PM


If one accepts second best then quite often that's just what one will receive

— Dag Hammarskjold

 

Current user of Rootsmagic version 7.5.7.0, Family Tree Maker 2014 and Legacy 7.5 on Win 10

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#9 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3060 posts

Posted 08 November 2015 - 02:21 PM

I have just uploaded a short screencast which will hopefully help some users who have expressed som confusion over Duplicate Search Merge in the past on these forums.

 

I watched Vyger's screencast with great interest. Duplicate Search Merge may be great in theory, but I concur that it's of no practical use in the real world for the reasons that Vyger cited. Being able to limit DSM to Named Groups would be a huge step forward, and I concur that being able to limit DSM to Named Groups would be of much greater value than being able to limit DSM to surnames. So I hope the RootsMagician will seriously consider Vyger's suggestions and also other improvements for DSM.

 

It is hugely important to me to be able to keep everybody together in one master database, and yet still be able to focus my attention on subsets of my database. This issue is so hugely important to me that when the ShareMerge feature first came out, I tried exporting a few dozen people into a temporary database, working on those few dozen people, and then importing them back into my master database with ShareMerge.  Here's the text that's associated with the ShareMerge option.

 

If you send a GEDCOM from your RootsMagic database to another RootsMagic user, they can import and make changes or additions to that file, and return it to you. You can import that modified GEDCOM into your database and use this option to automatically merge the two copies of the database back together.

 

I suppose that statement may be true in theory, but I find it not to be true in practice. In particular, it's actually a very manual process to merge the two copies of the database back together. That's because I have to examine every single individual that was merged to see what facts may have changed and therefore are now duplicated and determine which facts to keep. Hence, I find ShareMarge of no value and have sought other mechanisms to be able to work on subsets of my database in a practical fashion.

 

So let's return to Vyger's original point about the need to be able to limit Duplicate Search Merge to Named Groups, I would support him and go one step further and suggest that there is a huge need to limit nearly everything in RM to Named Groups. I think this is a much more important need for the Named Group facility than is the much discussed need for Named Groups to be dynamic. For example, why is People View the only main view that can be limited to Named Groups? Why can't RM Explorer and all its Finding and Marking and Unmarking dialogs be limited to named Groups. Why can't all the sidebars be limited to Named Groups? (If they could, we wouldn't need the Groups sidebar because the Index sidebar would automatically be a Groups sidebar when it was limited to a Named Group). And reports and exports could simply inherit the current Named Group that was being applied to the current view rather than having to specify a Named Group for each report.

 

Jerry



#10 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3183 posts

Posted 09 November 2015 - 06:05 AM

So let's return to Vyger's original point about the need to be able to limit Duplicate Search Merge to Named Groups, I would support him and go one step further and suggest that there is a huge need to limit nearly everything in RM to Named Groups. I think this is a much more important need for the Named Group facility than is the much discussed need for Named Groups to be dynamic. For example, why is People View the only main view that can be limited to Named Groups? Why can't RM Explorer and all its Finding and Marking and Unmarking dialogs be limited to named Groups. Why can't all the sidebars be limited to Named Groups? (If they could, we wouldn't need the Groups sidebar because the Index sidebar would automatically be a Groups sidebar when it was limited to a Named Group). And reports and exports could simply inherit the current Named Group that was being applied to the current view rather than having to specify a Named Group for each report.

 

Thank you Jerry and I agree completely regarding the need for an effective way to work within subsets of a database in all aspects, DSM was an easy example to make and within that feature there are many other enhancement opportunities still to be fulfilled.

 

I do worry a little when Bruce fulfils requests to export various RM customizations rather than the company selling the benefits of one database. Perhaps if our wishes are ever fulfilled that direction of maintaining one database, the benefits to users and the wider genealogy community will start to be encouraged.


If one accepts second best then quite often that's just what one will receive

— Dag Hammarskjold

 

Current user of Rootsmagic version 7.5.7.0, Family Tree Maker 2014 and Legacy 7.5 on Win 10

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#11 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 7919 posts

Posted 09 November 2015 - 12:45 PM

Confirming enhancements are in our tracking system.


Renee
RootsMagic

#12 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3183 posts

Posted 04 June 2018 - 06:27 AM

Would be nice to have a "do not merge" flag that could be set for any two individuals, to be referenced in subsequent merge operations - similar to the "not a problem" flag in the problem list.

 

The problems and concerns of Merging are again popular on Facebook and this forum so with RM8 in development I thought we should revisit this old wish.

 

I use Duplicate Search Merge a lot and Ludlow Bay made a nice point above which I effectively do today but in a long winded way. When I see a close match on DSM but cannot positively resolve I go into each persons edit person screen and add them to a group called "Resolve Work" for further examination and reconciliation later.

 

The way I use DSM is to merge around 10 individuals and then go back over the merged and open each person for edit and clean up. Obviously this is unnecessarily onerous on the user to keep track and not always necessary.

 

So a further thought, Rootsmagic could ADD any individual MERGED to a resolution group automatically and ONLY where the combined information count has increased, In other words those individuals with a perfect merge through ShareMerge or other would not require further reconciliation. This would overcome the very valid concern expressed by Jerry Bryan regarding these merged individuals, many of which now have duplication, becoming lost in the database.

 

A routine in Rootsmagic could also highlight those individuals with more than ONE same fact type within a user defined time span and add them to a Named Group, a script by TomH already achieves this to good effect for those able to use SQL.


If one accepts second best then quite often that's just what one will receive

— Dag Hammarskjold

 

Current user of Rootsmagic version 7.5.7.0, Family Tree Maker 2014 and Legacy 7.5 on Win 10

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#13 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 7919 posts

Posted 04 June 2018 - 08:36 AM

Confirming this is on the enhancement request list. 


Renee
RootsMagic