11 replies to this topic

[TomH]

This posting is in response to a request on the SQLite Tools wiki for a dynamically refreshed named group of "orphans". The user has a very large database with hundreds of these unconnected persons and wants to work through resolving them by using a named group.

Within RM, you can create such a named group by marking persons in the Mark Group dialog of RootsMagic Explorer with the criteria:

Number of children equals 0
AND
Number of sets of parents equals 0
AND
Number of spouses equals 0

However, as she connects, merges or deletes persons and because there are so many to begin with, she wishes that the group membership would be easily updated. It is pretty cumbersome to have to use RM's Edit Group functionality every time.

An alternative I suggested is to use Tools > Count Trees. Every unconnected person is a "Tree" with a count of 1 member. That's not terribly convenient because the list of trees is very long and is ordered in descending size, not by alphabetical or other sorting, and cannot be exploited as a Group can be by People View or any other report.

So I created another SQLite script that creates or maintains a group named "SQL: Unconnected Persons" and it's available for anyone to use without warranty of any kind at Group - Unconnected Persons.

And, once again, the plea goes out for the development of RootsMagic's Groups functionality to support the easy storing, editing and reapplication of a group's selection criteria.

[TomH]

And, once again, the plea goes out for the development of RootsMagic's Groups functionality to support the easy storing, editing and reapplication of a group's selection criteria.

And while they are at it, speed up the processing. It takes over two minutes to update a group using RootsMagic compared to 3 seconds for my script on a database of 160,000 with 1000 unconnected persons. A chunk of that is User Interface steps which would be dramatically shortened with a Refresh button right in the Groups sidebar. But a large part is Mark Group searching which is ponderously slow. Another is display updating which I have discovered is much faster in the People View of a 1000 member group than it is in the Sidebar - maybe by an order of magnitude so plenty of room for improvement there too.

[Laura]

I didn't see Number of siblings in the search criteria for the Group. Is there some reason to leave that out of the search criteria?

Add a user defined fact, Unconnected, to all pereon.s who will not be linked or merged and use Unmark, Unconnected exists is true to edit the Group.

Or, add a father, Unconnected, to those people and Unmark the descendants of Unconnected from the Group.

There is also the Individual report and choose the option for unconnected people. Save the report to a text file and open it in a word processor and delete each person from the text file as they are handled in the datebase.

The best solution might be exporting and importing a gedcom or drag and drop into a new database excluding the people with no connections. Database name, Connected

Select from a list:
Mark eveyone
Unmark using Tom's search criteria above

Perhaps create another database with only the person's excluded from the first new database. Database name, Unconnected. Drag and drop only the people that will be connected into the Connected database and link them.

Possibly, delete the people that are moved to the Connected database from the Unconnected database.

If Groups were to be made dynamic, we need either a way to have a choice of whether a Group will be dynamic or not when the Group is created or a Refresh button to recreate the Group from the original criteria.

[TomH]

Great suggestions, Laura.

I did not include the number of siblings because they are inherently 0 if there is no parent.

[Vyger]

And, once again, the plea goes out for the development of RootsMagic's Groups functionality to support the easy storing, editing and reapplication of a group's selection criteria.

I second that motion.

Once again great work by TomH on filling some of those Rootsmagic programming gaps.

[Renee Zamora]

Confirming enhancement requests are in our tracking system.

A while back I created a dummy family or all those orphaned people. The children were all connected to a fake father. Then I color coded everyone. I can now spot when I bump into them again in my database. I believe it was either Laura or Nettie that described that process a while back. It really works and has been extremely helpful.

I now have only 2 trees in my entire database. Not only are some really orphans there are small branches I don't know where they connect yet. There are 101 children in the fake family. I periodically check to see how many trees are in my database. That way I don't forget to link an orphan to the fake family. If you ever want to unlink them you just delete the family.

I went to Tools>Count Trees and clicked on Print. It gives you a list of all the trees and the record number of the starting person. That is what I used to quickly add "children" to my fake family.

[Renee Zamora]

And while they are at it, speed up the processing. It takes over two minutes to update a group using RootsMagic compared to 3 seconds for my script on a database of 160,000 with 1000 unconnected persons. A chunk of that is User Interface steps which would be dramatically shortened with a Refresh button right in the Groups sidebar. But a large part is Mark Group searching which is ponderously slow. Another is display updating which I have discovered is much faster in the People View of a 1000 member group than it is in the Sidebar - maybe by an order of magnitude so plenty of room for improvement there too.

Have you observed the same slowness right after running the File>Database Tools?

[TomH]

Integrity check was OK, over 70 secs to Rebuild Indexes.

Overall, the process might have been 30 seconds faster after rebuilding indexes but some of that was the shorter time I took with criteria setting - the three I used had been remembered from the previous run). The actual time taken for "Searching" or marking the persons was 33 seconds, 10x that of a SQLite query that does the same thing. I had not measured this separately earlier.

Once again, the sidebar index took quite a while to update while the People View of the group updated quickly; I cannot think of any reason why the sidebar list of the group should be slower than the People View of the same list; if anything, it should be faster as fewer tables and fewer fields are accessed and displayed.

Whether the marking took 3, 33 or 63 seconds, there remains the 60-90 seconds taken up by setting the criteria and waiting for the results to display. Lots of room for improvement and customer satisfaction...

[Vyger]

That is what I used to quickly add "children" to my fake family.

101 children is not prohibitive but processes such as TomH's groups do need to be automated, find criteria needs to be remembered by RM and the speed of processing needs to be improved, these are all becoming very old wishes.

Whether the marking took 3, 33 or 63 seconds, there remains the 60-90 seconds taken up by setting the criteria and waiting for the results to display. Lots of room for improvement and customer satisfaction...

Agree completely and such an easy thing to achieve, not forgetting refresh buttons on groups although it should also be refreshed by default when the group is first opened or switched to.

[Laura]

Agree completely and such an easy thing to achieve, not forgetting refresh buttons on groups although it should also be refreshed by default when the group is first opened or switched to.

Why would RM rerun and remark search criteria the first time a Group is first opened or switched to? Those same people would already have been marked when the group was created.

[Vyger]

Why would RM rerun and remark search criteria the first time a Group is first opened or switched to?

Might it be the Dynamic part of what we are talking about?

Those same people would already have been marked when the group was created.

Not if the those matching the find criteria has now changed.

[Renee Zamora]

Confirming issues and enhancements have been marked in our tracking system.