Jump to content


Photo

Some Thoughts on Dynamic Groups


  • Please log in to reply
14 replies to this topic

#1 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3606 posts

Posted 14 December 2010 - 11:57 AM

I've been using a Named Group to create sort of a "to do list" to identify individuals for whom I need to find 1930 census entries. As I find the entries and enter them into RM4, I've been refreshing the group manually to remove people from the named group for whom I have found their 1930 census data. My "manual refresh" does not consist of manually removing particular individuals from the group. Rather, it consists of manually rerunning the search criteria that was used to create the group in the first place. As such, I have some thoughts to offer on how this might work if/when Dynamic Groups become available.

First of all, the Named Group I'm describing is not particularly straightforward to establish. There are probably many other Named Groups that are easier to establish. The problem with this one is that if you search for "census year not equal 1930" as a way to find people without a 1930 census entry, you will get a match (for example) if the person has a census fact for both 1920 and 1930. The Find Dialog works just fine for facts that occur just one time, but it has major problems with facts such as census that naturally occur more than one time. So my basic approach in creating the group is as follows.

1. Do a "Mark Group" of "Everyone in the Database". Thereafter, I will be unmarking people for whom I don't need to find 1930 census data rather than marking people for whom I do. That's the only way I can think of to accurately find the correct people in this particular case.

2. Do an "Unmark Group" and "Clear people by data fields" with the following criteria: census date equal 1930 OR birth date after 1930 OR death date before 1930. The people who remain marked are my target population for whom I need to find 1930 census data.


In all truth, my "Unmark Group" operation includes a few additional tweaks, but the additional tweaks are not especially germane to this discussion. For example, I'm unmarking people whose color is not green. At the present time, I have a few thousand people who are colored green who are being "actively" researched, and I'm not presently interested in finding the 1930 census entries for anybody else.

Ok, so I do my thing and create my group. Using the group as a guide, I find and enter the 1930 census data for two or three families, say ten or fifteen people. And then I "refresh" my group to remove those ten or fifteen people from the group. Which is to say, I edit the group and repeat steps #1 and #2 above. The fact that I've added 1930 census facts for those ten or fifteen people causes the repetition of step #2 to remove them from the group.

Strictly speaking, step #1 shouldn't be necessary to repeat, but it doesn't hurt anything. Step #2 is trivial if I'm still in the same RM4 session because the same "census date equal 1930 OR birth date after 1930....." etc. search criteria is still there. If I'm not still in the same RM4 session, I have to re-enter the search criteria (and hope I still remember what it was).

"Remembering search criteria" is a wish list item that goes all the way back to Family Origins and to all previous versions of RM. The Named Group facility in RM4 seems to be targeted as the fulfillment of that wish. However, the initial implementation of Named Groups still does not fulfill that wish of remembering search criteria. It seems to me that no matter how automated or how manual the refresh of a dynamic group becomes in the future, the key element in the implementation has to be remembering the search criteria. I don't know if the RootsMagician might have in mind to be able to remember both my steps #1 and #2, or if he might have in mind just remembering step #2. I would only point out that I can easily envision situations where you need not only a step #1 and step #2, but you also might need a step #3, a step #4, etc. to properly define a group. My hope and wish would be that the number of steps that could be remembered would be quite large.

Back to my story. I create my group. I add some 1930 census facts to my database. I refresh my group to reflect the added census facts and to remove the people from the group who now have the additional data. The refresh consists of running steps #1 and #2 again. Step #1 is essentially instantaneous. Step #2 takes about 30 seconds for my computer to process. I exit the Edit Group dialog, and it takes the computer another 3 or 4 seconds to update the Named Group in the side panel. My database presently contains about 60,000 individuals. This is a fast, modern computer with plenty of memory (4GB), and no other program running except RM4.

Given how long it takes to refresh the group, I would question how wise it would be to routinely refresh the group on an automatic and "truly dynamic" basis. If the group were "truly dynamic", the computer could potentially spend all it's time just updating Dynamic Groups and I would never get any real work done.

Jerry

#2 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6263 posts

Posted 14 December 2010 - 01:05 PM

The request for a Refresh button has been posted before, in recognition of the potential load from automatically detecting changes and triggering a refresh. You give a great example of the sort of defining criteria that would suit Dynamic Groups.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#3 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 8489 posts

Posted 14 December 2010 - 01:45 PM

I have added Jerry's detail description in our tracking system's section on Dynamic Groups for further consideration.
Renee
RootsMagic

#4 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3431 posts

Posted 14 December 2010 - 01:50 PM

The request for a Refresh button has been posted before, in recognition of the potential load from automatically detecting changes and triggering a refresh. You give a great example of the sort of defining criteria that would suit Dynamic Groups.

Really each time a Group is opened it should be refreshed with the stored find criteria used to create it, a manual Refresh button would also be of benefit.

Customers should never be frustrated by things they cannot do.

 

User of Family Historian 6.2.7, Rootsmagic 7.6.2, Family Tree Maker 2014 & Legacy 7.5

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#5 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6263 posts

Posted 23 November 2011 - 10:31 PM

Still wondering and waiting... so I developed an outboard "Refresh" button for a particular (and maybe rather simple) group, but it demonstrates a principle around which a more complex set of rules might be cooked up. See Ancestors Named Group and Ancestors Query around which it is built.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#6 John James

John James

    Advanced Member

  • Members
  • PipPipPip
  • 222 posts

Posted 24 November 2011 - 10:11 AM

Named groups are currently static. We will be addressing dynamic groups (for lack of a better name) down the road a bit. We are having to temporarily remove some functionality that isnít ready yet in order to get the program out like everyone is clamoring for.


Still wondering and waiting...

If this is "down the road a bit", I wouldn't like to see something planned for the future. :angry:

#7 Laura

Laura

    Advanced Member

  • Members
  • PipPipPip
  • 4276 posts

Posted 24 November 2011 - 11:24 AM

Really each time a Group is opened it should be refreshed with the stored find criteria used to create it, a manual Refresh button would also be of benefit.


No! No! No!

As I have said before, a group should never automatically be refreshed. I have several groups that I search for a criteria and change something that can't be searched for. If one of those Groups is automatically refreshed, the people that I have manually unmarked would be marked again.

A manual refresh button would be useful.

#8 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3606 posts

Posted 25 November 2011 - 08:53 AM

No! No! No!

As I have said before, a group should never automatically be refreshed. I have several groups that I search for a criteria and change something that can't be searched for. If one of those Groups is automatically refreshed, the people that I have manually unmarked would be marked again.

A manual refresh button would be useful.

I totally agree that a group refresh should not destroy any manual customizations that you have made to a group. But it seems to me that it shouldn't matter whether the refresh was automatic or whether the refresh was manual. In either case, the refresh should honor the entire group definition no matter of whether the group definition was based on some kind of search, whether the group definition was based on manual "one at a time" check marks of people, or whether the group definition was based on both kinds of activities.

My model for this kind of processing is GenSmarts. GenSmarts has a facility akin to RM4's Named Group facility, except that GenSmarts calls it something else. The way GenSmarts accomplishes this magic is that for each "group" it has a little text file with the "group" definition. The first line in such a text file might say "include all the descendants of person #12847". The second line in such a text file might say "don't include person #5003" if you manually excluded person #5003. The third line in such a text file might say "include person #8977" if you manually included person #8977. The fourth line in such a text file might say "include descendants of person #2294". The descendants of person #2294 would be in addition to the descendants of person #12847. Etc.

Of course in the user interface within GenSmarts, you specify individuals by name rather than by number, but the little text file stores the record number for the individual of interest. Because the GenSmarts text file can include search rules such as "all the descendants of ...", one line in the file can represent thousands of people. Under normal circumstances, you never see the little text files that contain the group definitions nor do you even know they exist. You just specify your group creation rules to the GenSmarts user interface, and it creates the little text files for you behind the scenes.

I would wish for RM4 to include such capabilities. I would assume that RM4 would store such group definitions in SQLite tables rather than in text files.

With this kind of a capability in place, Laura's manual customizations would not be destroyed by a refresh, whether the refresh itself were automatic or manual.

Jerry

#9 Laura

Laura

    Advanced Member

  • Members
  • PipPipPip
  • 4276 posts

Posted 25 November 2011 - 11:24 AM

I unmark people I have already corrected from a group I created by using search criteria. These are not people I manually marked or unmarked whether using search criteria or not when I first created the Group.

I fail to see how automatically running my original search criteria would not remark the people I unmarked manually as having been done.

I don't want to have to go to the extra work of:

1. Have to remember where I left off.

2. Remember to Bookmark the next person to start working on every trims I quit the group.

3. Add a User defined fact to those I have done so I can use that fact to add to the search criteria thereby creating in essence a new group.

I'm not particularly eager to wait to start working on Group to rerun the search criteria every time I select it either.

#10 kbens0n

kbens0n

    Advanced Member

  • Members
  • PipPipPip
  • 3459 posts

Posted 25 November 2011 - 01:08 PM

I fail to see how automatically running my original search criteria would not remark the people I unmarked manually as having been done.

The key, maybe, would be in separating out group "result sets" from saved "grouping criteria". Automatic rerun of some saved "group criteria" could prompt user for either - overwrite of last "result run" (by named group perhaps) or offer to create a new result set (new named group or adding 1,2,3 to end of group name, or whatever). Could get unwieldy fast, though, with constantly changing database members. :)

---
--- "GENEALOGY, n. An account of one's descent from an ancestor who did not particularly care to trace his own." - Ambrose Bierce
--- "The trouble ain't what people don't know, it's what they know that ain't so." - Josh Billings
---Ô¿Ô---
K e V i N


#11 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6263 posts

Posted 25 November 2011 - 01:18 PM

I unmark people I have already corrected from a group I created by using search criteria. These are not people I manually marked or unmarked whether using search criteria or not when I first created the Group.

I fail to see how automatically running my original search criteria would not remark the people I unmarked manually as having been done.

You have to edit the group in order to unmark a member of the group. What if editing the group actually edited the script of rules by which the group was created? Then unmarking the corrected individual would add to the script of rules to "Unmark so-and-so". Next time you refresh the group, any newly qualifying members according to the rule algorithms get added, newly disqualified ones get deleted, and all manual mark and unmark rules from the original creation and subsequent edits get applied, all in the historic order that the rules were applied. Think of it as a macro recorder in Excel, your actions recorded in sequence for subsequent replay.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#12 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3606 posts

Posted 25 November 2011 - 06:54 PM

I fail to see how automatically running my original search criteria would not remark the people I unmarked manually as having been done.

The concept I'm describing would work as follows. Suppose you created a group with a search criterion that selected 100 people. Suppose a week later, you manually deleted one person from the group, leaving 99 people in the group. Suppose another week later you manually deleted another person from the group, leaving 98 people in the group. Finally, suppose that a month later you "refreshed" the group. The group definition should save the search criterion used to create the group. Of course, at this point there might be 103 people or 96 people or some such who now meet the original criteria. The group definition would also save the two people whom you manually deleted from the group. So the "refresh" would clear the group, search again to recreate the group based on the original criteria, and delete the two people you deleted manually from the group.

Jerry

#13 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3431 posts

Posted 26 November 2011 - 08:44 AM

I believe this discussion is now pointless as Dynamic Groups in addition to a lot of other features appear to be outside the will or the abilities of the programmers and I would like to echo the disappointment of other users.

We were promised more frequent updates and we received less frequent updates so RM4 seems to be low priority. I donít monitor this board as much these days as most has already been said and there is nothing new to talk about, also like some others I have engineered my own outside solutions to RMís shortfalls.

Regarding Named Groups I hear the fears of some users but there should be nothing to worry about as they are really quite simple. Firstly the STATIC group cannot really disappear as the criteria used to create it cannot be recreated, I can already hear calls that John Smith was there and now he is not. The DYNAMIC part can only apply to those groups created with a defined LOGIC, in other words a find criteria and something RM can recreate.


Each group would be flagged as STATIC or DYNAMIC, so itís one or the other, very simple, and that defines the action of any refresh. The status of the Group should also show in the header alongside Groups.

If an adjustment is made to a DYNAMIC Group by MANUALLY deselecting or selecting someone then that Group now becomes STATIC as the LOGIC is no longer sound.

DYNAMIC groups should REFRESH when selected/opened and a REFRESH button should also be available.

An EDIT button should also be available in the header for any DYNAMIC group to allow the user to EDIT the find criteria to better suit their needs.


Wishing for RM to remember MANUAL adjustments along side FIND criteria is not impossible but just gets too messy, either the user is in control of the Group or the Logic used to create it, not both.

This is not rocket science and I can only conclude that this is probably being held back for V5 despite it being ďtemporalityĒ withdrawn from V4.

Customers should never be frustrated by things they cannot do.

 

User of Family Historian 6.2.7, Rootsmagic 7.6.2, Family Tree Maker 2014 & Legacy 7.5

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#14 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6263 posts

Posted 26 November 2011 - 03:24 PM

More progress on demonstrating the feasibility of refreshable Named Groups, with the addition of a SQLite script for including persons lacking a Census fact for a user-specified year and jurisdiction AND incorporating the equivalent of manual marking or unmarking of members in the group. Full story at Named Group Refresh.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#15 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6263 posts

Posted 03 September 2013 - 09:07 AM

This topic really belongs in the RootsMagic Wish List, not in RootsMagic 4 > Discussion, because it is still unfulfilled in RootsMagic 6. Renee?

I recently created two new SQLite queries that create and update named groups, effectively resulting in a dynamic group within RootsMagic with an outboard Regenerate control. They are found at:
Group - Persons with Duplicate Events
Group - Persons with Text Dates

Previously published SQLite procedures related to Named Groups are listed at:
http://sqlitetoolsfo...tag=named group

Discussions in the RootsMagic Forums on this topic may be found under this Google search:
https://www.google.c....rootsmagic.com

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.