Jump to content


Photo

Conflict between files on Rootsmagic To-Go


  • Please log in to reply
34 replies to this topic

#21 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 5464 posts

Posted 22 February 2013 - 10:18 AM

Jerry, without reporting both the Created and Modified timestamps for both the removable and the computer drives, there is not enough information for analysis. You may well have hit upon the source of the glitch, though; RM2go having its own discrepancies in reading timestamps. So it would be wise to log what Windows file properties reports for all 4 timestamps along with what RM2go reports.

Tom user of RM7230 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> Rmtrix_tiny.png app, a growing bundle of RootsMagic utilities.


#22 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 2612 posts

Posted 22 February 2013 - 02:33 PM

Jerry, without reporting both the Created and Modified timestamps for both the removable and the computer drives, there is not enough information for analysis. You may well have hit upon the source of the glitch, though; RM2go having its own discrepancies in reading timestamps. So it would be wise to log what Windows file properties reports for all 4 timestamps along with what RM2go reports.


Trouble is, the RM-To-Go transfer utility does not display the Created date/time. I can definitely report the Created date/time from outside of the RM environment but apparently not from within.

The next time it happens, I will report six things: 1) Created date/time for Removable reported from outside the RM environment, 2) Created date/time for Hard disk from outside RM environment, 3) Modified date/time for Removable reported from outside the RM environmnet, 4) Modified date/time for Removable reported by RM-To-Go transfer utility, 5) Modified date/time for Hard disk from outside RM environment, 6) Modified date/time for Hard disk reported by RM-To-Go transfer utility.

The thing that struck me about the particular data that I did report was that the RM-To-Go transfer utility was off by a second or two on the Modified date/times as compared to the Modified date/times reported outside the RM environment (i.e., by another utility that is looking at the Windows folders). That seemed to me to be an error, irrespective of the Creation dates. But you are correct, the Creation dates do need to be reported.

Jerry

#23 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 2612 posts

Posted 22 February 2013 - 09:44 PM

Here's some new (more complete) data as requested by Tom. It took a while to get the conflict to happen. All dates are 2/22/2013, so I will only list times below. Remember that the RM-To-Go transfer utility does not display Create timestamps, so the only Create timestamps that are available are from outside of the RM environment.

It's probably obvious by looking at the times, but the scenario was to have everything all synced up on my main computer (the last successful sync was from the hard drive to the removable drive), to use the real RM on the main computer for a while, to shut down the real RM on the main computer, and to run the RM-To-Go transfer utility on the main computer. The desired outcome is that the RM-To-Go transfer utility would copy from the hard drive to the removable drive. Instead, a conflict is raised saying I've updated both the removable drive and the hard drive (I have only updated the hard drive). So I have to realize the program is in error and do a manual override.

Create time
6:24:10 PM removable drive as seen by utility program outside of RM
6:27:30 PM hard drive as seen by utility program outside of RM
Modified time
6:26:29 PM removable drive as seen by utility program outside of RM
6:26:30 PM removable as displayed by RM-To-Go transfer utility
7:16:56 PM hard drive as seen by utility program ouside of RM
7:16:58 PM hard drive as displayed by RM-To-Go transfer utility

Jerry

#24 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 5464 posts

Posted 23 February 2013 - 07:28 AM

Is there a typo on the Create times? How could the hard drive be later than the removable? It can be if the transfer was in the opposite direction, from removable to hard. It might be if there was a Restore done on the hard drive but I thought original Create times were preserved though a zip/unzip.

Is this a rather large database? Such that it takes about 0:2:20 to complete the transfer, i.e., the difference between the removable's Create and Modified times? If the Modified timestamp is the time at completion of transfer and the Created timestamp is that at the beginning of the transfer, then, if it is a non-zero difference that deems the file has been modified, RM2go will think it has been modified, ergo Conflict Create timestamp of the target is system time at the end of the transfer; Modified timestamp is that of the source file until opened by RM. Create and Modified timestamps of the source file are unchanged by transfer by RM2go.

The 1-2 second time discrepancy may be inconsequential given these other much grosser differences.

Tom user of RM7230 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> Rmtrix_tiny.png app, a growing bundle of RootsMagic utilities.


#25 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 5464 posts

Posted 23 February 2013 - 09:42 AM

Jerry, I went over this again with a large database file and think there is a procedural cause to your problem. I suspect you have opened both the file on the removable and the file on the hard drive since the transfer. Like Huygens' Uncertainty Principle, the mere act of observing modifies the outcome! When you merely open a file with RootsMagic, its Modified timestamp is changed because there is something written to the ConfigTable. So even though you have not deliberately changed any data, the file has been changed. I cannot otherwise account for why the Modified timestamp for your removable drive would be later than its Created timestamp, which is the system time at the end of the transfer.

Given two database files "synchronised" by RM2go, D1 and D2, C for Created timestamp, M for Modified

Transfer from D1 to D2:
C1 < C2, completed at time C2; M2 = M1 and M2 < C2.

If you open D2, then M2' > C2 and M2' > M1. RM2go detects that D2 has been modified and is the newer file, no conflict.

If you then open D1, then M1' > C2. RM2go detects that D1 has been modified. M1' > M2' indicating D1 is the newer but because D2 has also been modified, timestamps alone cannot resolve which one is primary, hence Conflict!

Thus the first file of a "synchronised" pair to be opened is the Master. If the other one of the pair is subsequently opened, RM2go cannot resolve which one is the Master.

It should be possible to avoid the Conflict by opening only one of the files between transfers.

Tom user of RM7230 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> Rmtrix_tiny.png app, a growing bundle of RootsMagic utilities.


#26 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 2612 posts

Posted 23 February 2013 - 02:27 PM

Is there a typo on the Create times? How could the hard drive be later than the removable? It can be if the transfer was in the opposite direction, from removable to hard.


I think I'm going to have to go through the whole scenario again to be sure, but I think the inconsistency in the Create times is not a typo in the Create times but rather that the last successful sync actually was from the removable drive to the hard drive and that I reported the direction of the last transfer backwards. I do know with great confidence that I only opened the hard drive file after the last successful sync and didn't open the removable file.

What I'm going to try to do (and it will take a while because the problem is not repeatable) is to make of screen capture video of the whole process from beginning to end. I think that's about the only way anybody (including me) is going to totally believe the scenaria I'm describing. :)

Jerry

#27 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 2612 posts

Posted 25 February 2013 - 11:20 PM

Every time I think I just about have this thing figured out, it throws me for a loop. The following timestamps look totally valid, but the RM-To-Go transfer utility reports a conflict. The sequence was: 1) sync from fixed to removable at 12:05:14 AM, 2) modify removable at 3:33:58 PM, 3) sync from removable to fixed at 6:00:44 PM, and 4) modify fixed at 11:44:34. This seems like the absolutely correct usage, and at this time the utility should sync from fixed to removable without reporting a conflict.

Jerry

			 Created		 Modified
Removable	 2/25/2013 12:05:14 AM		 2/25/2013 3:33:58 PM
Fixed		 2/25/2013 6:00:44 PM		 2/25/2013 11:44:34 PM


#28 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 5464 posts

Posted 26 February 2013 - 06:46 AM

I agree that the time stamps do not indicate a conflict. Does RM2go report the times correctly? Earlier you cited an incident where it got it wrong.

There's an (unrelated?) confusion in RM in which it replaces the default folder for database files with a path that it recently used for something else, such as saving a report. Do you have more than one .rmgc file of the same name on the hard drive?

Tom user of RM7230 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> Rmtrix_tiny.png app, a growing bundle of RootsMagic utilities.


#29 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 2612 posts

Posted 26 February 2013 - 08:55 AM

I agree that the time stamps do not indicate a conflict. Does RM2go report the times correctly? Earlier you cited an incident where it got it wrong.


I actually spent 8 or 10 hours working on this problem over the weekend. I've been waiting to report some of my observations until I feel like I have the whole picture, but let me go ahead and report one of the observations here. There are four timestamps of concern - the Created and Modified timestamps for the removable file and the Created and Modified timestamps for the fixed file. I found a tool that lets me change any or all of the four timestamps without actually opening or manipulating the content for either of the two files. I can thus manipulate the timestamps any way I wish and then start up the RM-To-Go transfer utility to see how it responds to various combinations of timestamps, valid and otherwise.

The RM-To-Go transfer utility is obviously looking at the Created timestamps, but it does not display them anywhere. So I have no direct way to tell if it is interpreting the timestamps correctly. I can only observe its behaviour when given various Created timestamps.

The RM-To-Go transfer utility does display both Modified timestamps. It always displays them correctly in the sense that it displays them within plus or minus two seconds of the value you can see in the Windows properties. Usually the utility shows the same timestamp as Windows, but sometimes it's off by a second or two. But when it displays the timestamps off by a second or two, its behavior is not adversely affected. Which is to say that despite being off by a second or two in the way it displays the timestamps, it is still declares the two files to be the same and does not perform a sync operation.

Just to be clear, I'm not talking about a one to two second difference between the Modified timestamp for the removable file as compared with the Modified timestamp for the fixed file. I'm talking about a one to two second difference between the Modified timestamp for the removable file as displayed by the transfer utility and the Modified timestamp for the removable file as displayed by Windows (and similarly for the fixed file). There is not always a difference, but sometimes there is. Arguably, it could be Windows instead of the transfer utility that's off by a second or two in its display, but I have a file maintenance utility that shows timestamps and it always matches Windows. I've not gotten to the point (yet, hope I don't have to!) of dumping out the timestamps in hex and doing the conversions myself.

To carry this observation a little further, no matter how screwed up I make the Created timestamps the transfer utility correctly does nothing if the Modified timestamps match. The transfer utility seems to consider the timestamps to match if there are the same within at most two seconds. This time, I'm talking about a difference beween the fixed and removable timestamps, not a difference in the display of one of the timestamps. At a difference of three seconds or more, the Modified timestamps are no longer considered to match by the transfer utility and it then considers the Created timestamps when it is deciding what to do.

Just to emphasize: the Created timestamps are not even looked at by the transfer utility unless there is a mismatch in the Modified timestamps. For example, if the Modified timestamps match within two seconds and the Modified timestamps are later than both Created timestamps, then no conflict condition is raised.

Jerry

#30 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 2612 posts

Posted 26 February 2013 - 09:15 AM

There's an (unrelated?) confusion in RM in which it replaces the default folder for database files with a path that it recently used for something else, such as saving a report. Do you have more than one .rmgc file of the same name on the hard drive?


I've actually been a little worried about this issue. The answer to the question you asked is: no, I do not have more than one .rmgc file of the same name on the hard drive.

But in answer to a question you didn't ask, I do have more than one .rmgc file of the same name on the removable drive. Therefore, if I had some sort of procedural error where I used the wrong file on the removable drive or where RM used the wrong file on the removable drive or where the RM transfer utility used the wrong file on the removable drive, then I could run into a sync problem.

The reason for more than one .rmgc file of the same name on the removable drive is that I'm using the same removable drive with RM-To-Go and also as a backup for about about 20GB of data from my hard drive. (The same 20GB is also backed up in several other ways, so the removable drive is not the only backup.) The folder used by RM-To-Go on the removable drive is totally separate from the folder used for backup purposes. When using RM and when using Rm-To-Go I'm always very careful to confirm that I'm using the correct folder. I'm not aware of any time that I've used the wrong folder (well, I probably wouldn't be aware of it if I made a mistake, would I?). But the conflict problem with the RM-To-Go transfer utility is such a common and persistent occurence that I figure I couldn't possibly be making that many procedural mistakes on a near daily basis.

Jerry

#31 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 2612 posts

Posted 04 March 2013 - 12:13 PM

But in answer to a question you didn't ask, I do have more than one .rmgc file of the same name on the removable drive. Therefore, if I had some sort of procedural error where I used the wrong file on the removable drive or where RM used the wrong file on the removable drive or where the RM transfer utility used the wrong file on the removable drive, then I could run into a sync problem.


I have arrranged to eliminate the situations where I had more then one .rmgc file of the same name on the removable drive. Doing so did not change the problem. The RM-To-Go transfer utility still sometimes (really, more than half the time) reports a sync conflict when in fact there isn't a sync conflict. And the problem remains to be not repeatable in any sort of reliable way. I still don't know the precise conditions that cause the problem.

Jerry

#32 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 5464 posts

Posted 30 November 2015 - 02:14 PM

This problem with false "Conflict" in RM2Go has become a topic on the Facebook group. It appears that the RM7 version still suffers from what Jerry reported on with RM5. Except it seems to be highly repeatable. Merely open/close either file after a transfer from removable drive to system drive and the result is a Conflict warning when there should not be. The open/close operation modifies the database thus changing the file's Modified time stamp (no need to edit anything to effect the changed time). The complementary operation of open/close either file after a transfer from system to removable does not result in a conflict warning and that is as it should be. So the opposite behaviours are logically inconsistent - only one can be right.

 

I delved into it further trying to devise an algorithm that could decide what permutations of the four file time properties (Created and Modified for the two files being compared) indicated non-conflicting vs conflicting. I have not figured it out and maybe it is beyond my powers to do so! However, based on time stamps alone, there is nothing absolute about the decisions. There is a third condition that lies between probable conflict and probable non-conflict --- ambiguous! --- and that is really tough given that there are 256 permutations possible. My mind is turning into mush.

 

Probably non-conflicted: (in order of diminishing probability)

  1. If the Modified time values are identical, then these are identical files to a very high probability. It is highly unlikely that two different files could have been saved at precisely the same time on two different computers.
  2. If the file with the higher Created also has a higher Modified and its Created is higher than the other's Modified, then it is a probable transfer from the other and was subsequently modified. This later Created and later Modified file is probably the master. The certainty of this is in the mind of the user as it is possible to create this condition with two very different files.
  3. If the file with the higher Created also has a Modified lower than its Created and lower than the other's Modified which itself is later than the higher Created, then the higher Created was a probable transfer from the other (original), has not been modified since then while the original has been modified since. The later Modified is probably the master. The certainty of this is in the mind of the user as it is possible to create this condition with two very different files.

 

Probably conflicted:

  1. If the Modified times differ and both are later than the later Created time, then both files have been modified since the last transfer. That could be as simple as both having been opened/closed with no change in the user data but the user will have to decide which one is to be master.
  2. If the Modified times differ and both are earlier than the earlier Created time, then both files have been transferred since last modified suggesting they traversed different routes and may not have even started from a common file. 

Ambiguous:

  1. Dare I say every other combination? I'm unsure. There are some combos that just will not occur in practice. And some more that cannot if the user has scrupulously maintained their files in but one location on the host drive and but one location on the only removable. And there are confounding factors that may arise by restoring over a file or if the host location is also a DropBox or other synchronizing folder.

That said, I have wasted far too much time digging into the illogical behaviour of a tool that I don't even use or use, rarely, at most.


Edited by TomH, 30 November 2015 - 10:19 PM.

Tom user of RM7230 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> Rmtrix_tiny.png app, a growing bundle of RootsMagic utilities.


#33 c24m48

c24m48

    Advanced Member

  • Members
  • PipPipPip
  • 2612 posts

Posted 30 November 2015 - 02:33 PM

For a variety of reasons, I have gone from using the tool frequently to using the tool rarely. Nonetheless, it's interesting that the problem is being reported again after all this time.

 

To the best of my understanding, RM never accepted that there was a problem. But IMHO, there definitely is a problem. It's the kind of problem that I think should be addressed as a super high priority, even if not many people are encountering the problem. That's because the problem presents a very high possibility for data loss.

 

Jerry

 



#34 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 5464 posts

Posted 30 November 2015 - 03:33 PM

To the extent that the RootsMagician participated in the Facebook discussion (and he does appear there more frequently than here or the old ROOTSMAGIC-USERS mailing list), he has not acknowledged there is a problem.

 

I'm not sure about the risk of data loss. I guess the user could make the wrong decision based on the only info presented (which one is later and which one is larger). It might be more informative if the RootsMagic Database Properties window could be opened on both databases for side-by-side comparison, with color-coding to flag which property is larger. Some added info would be good, such as the latest "Date last edited" record property which is probably of more significance than the File Modified property which is affected merely by looking at the database.

 

Then again, the lack of complaints may indicate the degree to which the feature is used or perhaps a pattern of usage that somehow obviates the false conflict.


Tom user of RM7230 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> Rmtrix_tiny.png app, a growing bundle of RootsMagic utilities.


#35 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 5464 posts

Posted 30 November 2015 - 04:08 PM

Renee confirmed that she saw the problem in 2010 but that might have been before the "tracking system". I have not found if/when she added it to the "tracking system" as a bug.

Early reports:


Tom user of RM7230 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> Rmtrix_tiny.png app, a growing bundle of RootsMagic utilities.