Jump to content


Struggling with Database Clean-up

cleanup ancestry duplicates media

  • Please log in to reply
4 replies to this topic

#1 HowdyPez



  • Members
  • PipPip
  • 5 posts

Posted 15 January 2019 - 07:57 PM

So, I've made a mess of things by keeping my tree on Ancestry for so long [well, not entirely my fault, Ancestry shares in the blame].  I purchased RM about 9-10 months ago to get back to doing things the 'right way'. However, I am running into several issues and I don't know if RM doesn't have that feature/function or if I'm missing something.


Here goes:


1. Ancestry has multiple facts listed for each person [mostly census records]. Is there a way to merge duplicate facts? If not, is there a way to delete more than one fact at a time [to delete duplicate facts for each person in my database one at a time would probably cause me to just start a database from scratch].  The facts are also tagged twice in the media file [again, mostly the census media items].


2. I apparently started out sourcing properly at one time, but then 'fell off the wagon' and In my database I have several repositories that are essentially the same from the completely unhelpful "Have a copy" to "My Files" to a proper "McKay Research Files".  Is there any way to merge them?  Or can I print all the sources that have "have a copy" so that I can change the repository to the proper one and then delete the horribly named one?


3. Media files - I know, I know so frustrating.  My issue [aside from having to rename over 800 files] is that almost all of them have a duplicate!  Is there a way [either in the master Media Gallery or the person's media gallery] to merge or delete duplicates?


I had some other questions, but they basically revolve around the need to have a merge tool where there isn't one or a multiple selection tool for deleting [on windows a Ctrl+click to select several things to delete at once].


I do have screenshots of my issues, but couldn't figure out how to attach them to this post.  If none of this is possible and it all needs to go to the wish-list area, just let me know :)  


Thanks in advance for any assistance,




P.S. if anyone knows of a service that I can pay for where they will clean up my database and sources, I'd like that information as well LOL!  ;)


#2 TomH


    Advanced Member

  • Members
  • PipPipPip
  • 6449 posts

Posted 15 January 2019 - 09:35 PM

No, No, No. Sorry. Whether it's on Ancestry or in RootsMagic, it's one at a time or not at all. And doing it on one side of TreeShare does nothing to the other side until you deal with those differences, one by one. Some things may be batched by operating on the database directly using SQLite, e.g., reassigning replica repositories to one master. 


As for a cleanup service, what are you willing to pay? And what are your priorities? The effort involved is impossible to estimate so you may get what someone might accomplish in so many hours, e.g., what you're willing to pay divided by their hourly rate. And their hourly rate may reflect how capable and effective they may be. Where does one shop for such services? Maybe the best bet is to hire a family tree consultant to build a new, systematic database from the material you have and additional research they may do on your lines of interest with the result being an evidence-driven tree and a database you can expand on.

Tom user of RM7630 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.

#3 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3982 posts

Posted 16 January 2019 - 07:29 AM

I think that in your situation, the best strategy would be to start over. But I wouldn't start over from scratch. I would choose one of the ancestry files - whichever one seems to be in the best shape, and use TreeShare to transfer it to RM. I would then cleanup that RM database. Finally, I would manually look at the other ancestry databases for information that might otherwise be missing. I would enter that information into my now one and only RM database, but I wouldn't merge any of the other ancestry databases in any sort of automated fashion. I would just use them for reference. Finally, I would disconnect my RM database from ancestry. I would either just leave it that way, or else use TreeShare to make make a new ancestry tree.


So I would have a new standalone RM database or else a new RM database  + a new linked ancestry tree. At that point, I would either leave the other ancestry trees there for occasional reference or else I would just delete them. This sounds like a drastic solution, but I think it's the only sort of thing that's likely to really work.



#4 Vyger


    Advanced Member

  • Members
  • PipPipPip
  • 3560 posts

Posted 16 January 2019 - 08:30 AM

No, No, No. Sorry.


Age old problems and little help from Rootsmagic over the years.


The problem with outsourcing a solution is that you would have to clealy define exactly what you want.


If I were tackling this problem I would combine everything into one NEW reference database and firstly run all Rootsmagics Auto Merges (check the file properties before and after), I would then work to make events, repositories exactly the same in all respects and then try an Export to Gedcom followed by an Import into another NEW file of a different name and see what Rootsmagic cleans up.


Cleaning the Place List is one important area as any slight differences even in punctuation will result in duplication.


I noticed some missing partners on my database recently and knew they were somewhere, I decided to combine all the gedcom files and split RM databases I have received over the years into one file. The combined file had more 1.3 million individuals and I ran all automatic merges (I trust this process) It was running on my laptop (disconnected from the internet so Microsoft couldn't disturb it with an update) and the process took 9 days to complete. The result was a file with 156K individuals which I keep to the side for reference and dragging and dropping, the file has loads of event and place duplication but if I find a missing spouse which I feen should be there I know it will that combined reference file.

Keeping ones customers and their important views at a distance is never a good approach


User of Family Historian 7.0.3, Rootsmagic 7.6.5


Genealogy software comparison and how to videos



#5 HowdyPez



  • Members
  • PipPip
  • 5 posts

Posted 16 January 2019 - 07:08 PM

Thanks for all the feedback, this is what I was afraid of.


I have run all the auto-merge functions I could find and did many more manual merge's where available [and/or not caught by the auto-merge].


I wish Ancestry had some more functions [like finding unattached trees] so I could delete those folks. I only have 2 trees in Ancestry - one maternal and one paternal, so deciding which to use isn't necessary.  I really want to avoid pulling my trees off of Ancestry and uploading a new corrected one [just out of the absolute dread of all those stupid hints!].


Tom, I like your idea of a 'tree consultant' I think I've seen that option on Genealogy Coach [which I've used before].  I might start with the media for sources and get those renamed and fixed [deleting the duplicates] so that I and my consultant know what I actually have  :blink:


Again, thanks for all the responses!!