Jump to content


Photo

DataClean Beta Feature

dataclean

  • Please log in to reply
67 replies to this topic

#1 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 4152 posts

Posted 21 February 2013 - 11:15 AM

There is hidden inside of RootsMagic 6 a utility that never came out of the beta stage. This feature is called DataClean. Currently there are two sections: NameClean and PlaceClean. It is not ready for prime time. But, some users have found it and are using it. The only way it can be accessed is by customizing the toolbar and adding the DataClean icon to it.

If you want to use it go ahead, but be patient with it, like we said it's not ready for prime time.

So we are not inundated with support tickets on features that need tweeking or enhancements that need to be made, we ask that you post your comments on this thread. I will monitor the discussions and made sure they are reported back to development.

Listed below are a couple of threads that discuss some issues found with DataClean. You may want to check these threads before commenting on something already reported.

Some Feedback on the DataClean Beta
http://forums.rootsm...dataclean-beta/

Data Clean-up
http://forums.rootsm...-data-clean-up/
Renee
RootsMagic

#2 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 2374 posts

Posted 21 February 2013 - 03:40 PM

You may not be aware that you can do live edits directly on the suggested "Cleaned Name" area. You do not have to accept what is suggests, you can manually change it to whatever you would like. Once you get the name the way you want it then hit Accept.


I seem to be the only one plugging this but maybe I am the only one who has inherited unrecognisable Text Dates. <_<

Rootsmagic already has the facility to report Text dates under Fact Type > Print however I believe DateClean should be part of this suite, easily achieved and combined with the existing manual edit would be a great way to report and clean any Text Dates into recognisable RM format.

I must also applaud the Rootsmagician for his attempts to clean up genealogy data, it's a pain we all know even from our own bad habits of the past, (well maybe just me :D ) and any such utilities are of great benefit.

Software Comparisons - Place Management - How other software packages stack up.
Media Gallery (a critical look) - Written when RM4 was introduced but still applies today.

Relaxation is the key to life and this is where I get some time to relax and catch up on my hobby and research s the key to life and this is where I get some time to catch up on me genealogy work and research


#3 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 4152 posts

Posted 22 February 2013 - 12:27 PM

Confirming enhancement request is in our tracking system.
Renee
RootsMagic

#4 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 2374 posts

Posted 23 February 2013 - 02:35 AM

OK, I have had a little time to compile some thoughts and mock ups to represent them. I do my fair share of cleaning files but I must admit not usually names so whilst I can see potential problems I will leave that to other users.

Firstly I would like to see the option to enter a comma delimited list of Place Detail Identifiers, this might just be Church or Hospital but I notice in my database Cemetery prior to the first comma in one Place is not highlighted as a potential Place Detail. I would see the users comma delimited entries being used to highlight any Place which contains any of those entries prior to the first comma and for my example I have chosen the German language to broaden Rootsmagics international appeal :D

Posted Image

Secondly I have borrowed this idea from another program, it's two little buttons which save a lot of typing or copy & paste. Clicking on each one moves the first comma delimited section either up or down depending on your wish. So in this example clicking the down button once would remove Andersons Bay Cemetery from the Place and put it in the Place Details field. Clicking the down button again would put add ", Dunedin" to the Place Details leaving only NZ in the Place. Clicking the up button would reverse this step by step.

Posted Image

Lastly there is no facility within RM to easily move the commonly found non place data to the description field and I know from personal experience that this is a common need. So an extra field to allow the correcting of non Place data would be beneficial and in this case would leave the Agricultural Labourer. as an Unused Place.

Posted Image

Software Comparisons - Place Management - How other software packages stack up.
Media Gallery (a critical look) - Written when RM4 was introduced but still applies today.

Relaxation is the key to life and this is where I get some time to relax and catch up on my hobby and research s the key to life and this is where I get some time to catch up on me genealogy work and research


#5 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 2374 posts

Posted 23 February 2013 - 03:43 PM

I just spent half an hour running through a third party file of 11K individuals using NameClean and thought I would share my observations.


* Numerous records were highlighted for Improper Capitalisation due to having a "Mc" derivative in the forename and this is very common in Scotland and elsewhere in the world. NameClean does not highlight this as an exception when in the Surname.

* There were a number of entries where the compiler had the Married Surname in [square brackets] in the Suffix Field, obviously the Maiden Surname was unknown. I would have liked the Alternate Name (third pane) opened here or at least have had the option to open it.

* On many cases I got the "Wife Shares Husbands Surname" exception and again I would have liked to have the Alternate Name pane opened or an option to open it in these cases.

* This has been stated elsewhere by other members but there were various times I wanted an Edit Person button to make the required changes to my satisfaction. The addition of an Edit Person button in the best solution here but at lease the Record Number alongside the name in the top pane would be helpful.

* I also came across many cases of improper punctuation where Character, Space, Period were entered and RM's suggestion here was to add a further Period rather than remove the Space.

* I came across a number of individuals named DEAN and RM constantly assumed this as a Prefix so again an option to Allow Dean as Forename.



I usually do this cleaning outside RM, mainly out of necessity in the past but this was quick to use and a very welcome addition within the program. What I do feel a need for is a few custom rules or even more checkboxes, whatever works best and I did feel that a preview report of what RM was proposing would have been nice to help set up such rules like a Before & After report.

I do hope these cleaning utilities come to maturity in the not too distant future, they are a very welcome addition within RM and cleaner data will benefit the genealogy community as a whole. :)

Software Comparisons - Place Management - How other software packages stack up.
Media Gallery (a critical look) - Written when RM4 was introduced but still applies today.

Relaxation is the key to life and this is where I get some time to relax and catch up on my hobby and research s the key to life and this is where I get some time to catch up on me genealogy work and research


#6 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 1211 posts

Posted 23 February 2013 - 11:19 PM

* I came across a number of individuals named DEAN and RM constantly assumed this as a Prefix so again an option to Allow Dean as Forename.


There are likely to be other names that have the same issue. Doctor and Major come to mind immediately, but surely there are several others.

Jerry

#7 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 2374 posts

Posted 23 February 2013 - 11:26 PM

There are likely to be other names that have the same issue. Doctor and Major come to mind immediately, but surely there are several others.

Jerry


Jerry, do you know anyone who actually has the forename Doctor or Major they would be new to me. Yes RM would suggest these for the prefix field and if they are valid forenames they should also have an exclusion clause.

Software Comparisons - Place Management - How other software packages stack up.
Media Gallery (a critical look) - Written when RM4 was introduced but still applies today.

Relaxation is the key to life and this is where I get some time to relax and catch up on my hobby and research s the key to life and this is where I get some time to catch up on me genealogy work and research


#8 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 1211 posts

Posted 23 February 2013 - 11:35 PM

Jerry, do you know anyone who actually has the forename Doctor or Major they would be new to me. Yes RM would suggest these for the prefix field and if they are valid forenames they should also have an exclusion clause.

I don't know anybody personally, but I have several men named Doctor in my database (and yes they were named Doctor - it wasn't some kind of honorific). And I know of at least two American football players who have been named Major.

As a good test of this idea, check any kind of birth record database at ancestry.com. At birth, nobody would be a medical doctor or a Ph.D. doctor yet, and nobody would be a major in the military yet. So Doctor or Major would really be the baby's first name.

Jerry

#9 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 2374 posts

Posted 23 February 2013 - 11:40 PM

My goodness I had never come across those as forenames so I have learnt something, thanks.

In one of my place mock ups above I suggested a field where users could enter Place Detail Identifiers and that would be useful in non English countries, I suppose a comma delimited list for things to ignore in forenames might also be a good universal suggestion.

Software Comparisons - Place Management - How other software packages stack up.
Media Gallery (a critical look) - Written when RM4 was introduced but still applies today.

Relaxation is the key to life and this is where I get some time to relax and catch up on my hobby and research s the key to life and this is where I get some time to catch up on me genealogy work and research


#10 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 1211 posts

Posted 23 February 2013 - 11:42 PM

In the NameClean and PlaceClean windows, there is a bottom pane of the window titled Problems to Fix with problem areas listed and checkboxes. How are those supposed to work?

Well, I've only been searching for one type of problem at a time, and I assume that the Problems to Fix pane is for the case where you are searching for more than one type of problem at the same time to show you which problem or problems the current name or place might have. But why the checkmarks? What happens if you turn them off? And why is an item called Description always in the list? Description is not an item you can select in the main NameClean window or PlaceClean window.

Just curious.

Jerry

#11 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 2374 posts

Posted 23 February 2013 - 11:44 PM

If RM makes several suggestions and I have seen three where a third pane popped up for Alternate Name, unchecking any of those boxes unapplies that suggested change.

Software Comparisons - Place Management - How other software packages stack up.
Media Gallery (a critical look) - Written when RM4 was introduced but still applies today.

Relaxation is the key to life and this is where I get some time to relax and catch up on my hobby and research s the key to life and this is where I get some time to catch up on me genealogy work and research


#12 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 4152 posts

Posted 28 February 2013 - 10:38 AM

Confirming issues and enhancements have been added to our tracking system.
Renee
RootsMagic

#13 Christine Frawley Hill

Christine Frawley Hill

    Member

  • Members
  • PipPip
  • 7 posts

Posted 01 March 2013 - 05:19 PM

I'd like to see the name of the person whose record I am changing. Also, there should be an option to back up the file before cleaning it up. A report of all changes has already been requested but I'd like to second that suggestion :-)

#14 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 2374 posts

Posted 04 March 2013 - 05:20 PM

My further trials of Place clean show me that in a number of cases I am left with identicle duplicate Places in the Place List after using it.

Prior to RM4 and in other genealogy software these identical Places and automatically merged and I now believe that as a final step to PlaceClean Rootsmagic should do the same and save the user further work.

Software Comparisons - Place Management - How other software packages stack up.
Media Gallery (a critical look) - Written when RM4 was introduced but still applies today.

Relaxation is the key to life and this is where I get some time to relax and catch up on my hobby and research s the key to life and this is where I get some time to catch up on me genealogy work and research


#15 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 4152 posts

Posted 04 March 2013 - 06:07 PM

Issue noted in tracking system.
Renee
RootsMagic

#16 Elynn

Elynn

    Advanced Member

  • Members
  • PipPipPip
  • 101 posts

Posted 11 March 2013 - 10:39 AM

My goodness I had never come across those as forenames so I have learnt something, thanks.

In one of my place mock ups above I suggested a field where users could enter Place Detail Identifiers and that would be useful in non English countries, I suppose a comma delimited list for things to ignore in forenames might also be a good universal suggestion.


I've got King, Queen, Princess and many with name of states - (Tennessee, Virginia, Georgia, Louisiana, Florida). I even have a Charles Census - because he was born on the day the census taker came to the house.

#17 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 1211 posts

Posted 13 March 2013 - 08:15 AM

There are several improvement for DataClean that all might be said to come under the general umbrella of making it possible to break a big job into smaller jobs. Some of the improvements have been mentioned before, and some have not. Here's at least a partial list of such improvements that come to mind.
  • The ability to stop a DataClean operation and resume where you left off. This capability would come to you automatically if you were able to accept every suggested cleaning, but sometimes you need to skip some of the suggested cleanings - at least temporarily, and sometimes forever.
  • The ability to have a "not a problem" list for DataClean operations. This ties in with #1.
  • The ability to apply a DataClean operation to something less than your entire data base, such as to everybody color coded blue or to everybody in a particular Named Group. For example, I might want to put everybody that I'm working on for an upcoming family reunion into a Named Group and then to run a DataClean operation just on them. Or I might want to run a DataClean operation just on the individuals in my tree. Etc.
Jerry

#18 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 1211 posts

Posted 13 March 2013 - 08:22 AM

As an anti-clickiness feature, Cancel should actually Cancel. It shouldn't produce a pop-up asking me if I want to Cancel.

There are other places in RM where a Cancel operation (or a Cancel-like operation) really does need a pop-up reminder that something important is about to be lost. But this problem doesn't exist for DataClean, so here is an opportunity to stamp out one little bit of excessive clickiness.

Jerry

#19 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 4152 posts

Posted 13 March 2013 - 11:47 AM

As an anti-clickiness feature, Cancel should actually Cancel. It shouldn't produce a pop-up asking me if I want to Cancel.

There are other places in RM where a Cancel operation (or a Cancel-like operation) really does need a pop-up reminder that something important is about to be lost. But this problem doesn't exist for DataClean, so here is an opportunity to stamp out one little bit of excessive clickiness.

Jerry


I think the Cancel pop-up should remain until we add a feature to resume where you left off, etc.
Renee
RootsMagic

#20 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 4152 posts

Posted 13 March 2013 - 11:51 AM

Confirming enhancement requests are in our tracking system.
Renee
RootsMagic





Also tagged with one or more of these keywords: dataclean