30 replies to this topic

[devonroots]

When updating my Rootsmagic to-go from the main computer, I sometimes get a 'conflict between files' note, which stops me from transferring the files.  Is there any way I can find out what the conflict is, or transfer the file anyway?

Also, can I get the To-Go to remember where I keep my files on my computer as I always have to click on the 'change folders' option?

[Renee Zamora]

A conflict happens when you have used both the desktop version and the RM To-Go version since the last sync. You need to keep track of which version is that latest. If you start using the To-Go version make sure you transfer the file over to the Desktop before you use that version.

For your folders:
Inside RM go to Tools>Program Options>Folder to set your default folder locations. Make sure all you files are in that folder. Then when using the RM To-Go utility click on Reinstall RootsMagic on (Drive) on the bottom left is "Show Options" click on that.  Check the box to install "My RootsMagic Settings". Reinstall.  It should then remember where your files are located.

[c24m48]

Renee Zamora, on 18 February 2013 - 10:35 AM, said:

A conflict happens when you have used both the desktop version and the RM To-Go version since the last sync.

I respectfully disagree. A conflict can happen as you describe. A conflict usually happens because of a bug in the date/time calculations in RM To-Go. RM To-Go calculates that the desktop version and the To-Go version have both been used since the last sync when if fact only one of them has been used since the last sync. In my case, the conflict is because of the bug at least 99% of the time that there is a conflict.

One possible bug in RM To-Go's date calculation is incorrect handling of FAT file systems on a removable device (usually a FAT32 file system these days). FAT/FAT32 file systems only store the date/time to the nearest two seconds. So to handle FAT/FAT32 file systems correctly, RM To-Go would have to allow for a plus or minus 2 second difference in the times since the last sync and still declare that only one of the two systems has been used since the last sync.

However, I have switched away from a removable device that uses FAT/FAT32 to a removable device that uses an NTFS file system, and the bug persists. I just have to override the conflict manually,and I have to do it nearly every time I sync.

If priority is ever given to fixing this bug, the development team has my sympathies. Those kinds of date/time calculations are a real beast in general, and to do the syncing correctly the calculations are probably going to have to deal with approximate equality rather than with exact equality.

Jerry

[c24m48]

I should mention that although this issue was brought up in the RM5 forum, I am running the latest version of RM6 and the bug is still there.

Jerry

[devonroots]

The program options show where my files are located, I did the reinstall after clicking on the settings button but nothing happened.  It still thinks my files are in My Documents and not in a sub-folder.

[Renee Zamora]

devonroots, on 18 February 2013 - 11:11 AM, said:

The program options show where my files are located, I did the reinstall after clicking on the settings button but nothing happened.  It still thinks my files are in My Documents and not in a sub-folder.

Try deleting the files off the flash drive. Then move the files using RM To-Go to the flash drive. Maybe it will remember the settings then.

[Renee Zamora]

c24m48, on 18 February 2013 - 10:53 AM, said:

I respectfully disagree. A conflict can happen as you describe. A conflict usually happens because of a bug in the date/time calculations in RM To-Go. RM To-Go calculates that the desktop version and the To-Go version have both been used since the last sync when if fact only one of them has been used since the last sync. In my case, the conflict is because of the bug at least 99% of the time that there is a conflict.

One possible bug in RM To-Go's date calculation is incorrect handling of FAT file systems on a removable device (usually a FAT32 file system these days). FAT/FAT32 file systems only store the date/time to the nearest two seconds. So to handle FAT/FAT32 file systems correctly, RM To-Go would have to allow for a plus or minus 2 second difference in the times since the last sync and still declare that only one of the two systems has been used since the last sync.

However, I have switched away from a removable device that uses FAT/FAT32 to a removable device that uses an NTFS file system, and the bug persists. I just have to override the conflict manually,and I have to do it nearly every time I sync.

If priority is ever given to fixing this bug, the development team has my sympathies. Those kinds of date/time calculations are a real beast in general, and to do the syncing correctly the calculations are probably going to have to deal with approximate equality rather than with exact equality.

Jerry

Jerry did you check that the computers clocks are in sync between each computer used.

[c24m48]

Renee Zamora, on 18 February 2013 - 11:21 AM, said:

Jerry did you check that the computers clocks are in sync between each computer used.

Renee Zamora, on 18 February 2013 - 11:21 AM, said:

Jerry did you check that the computers clocks are in sync between each computer used.

My computers are always kept in sync with atomic clocks on the Internet. If they differ, it would be by much less than a second.

Jerry

[c24m48]

I've been fighting this problem and thinking about it ever since RM4 was released. It seems to me that there have to be at least three date/times involved in the problem, and maybe four. One would be the "date/time last modified" in the Windows folder for the copy of my RM database on my C: drive. The second would be the "date/time last modified" in the Windows folder for the copy of my RM database on my removable drive. And the third would be the "date/time last synced", but stored where?

I've never really been able to figure out where the "date/time last synced'" is stored. I've looked inside the RM database itself as well in various INI and XML files associated with RM. If knew where it was stored, I would be happy to fight through the awful date/time calculations that are associated with the way Windows stores date/times internally to look at all three date/times and see if I can figure out exactly where things are going wrong.  No doubt, TomH already knows where the required information is stored and is going to post the answer within five minutes of when I post the question.

I suggested that there might be four date/times involved in this problem. That's because the "date/time last synced" might be stored twice depending on where exactly it's stored. Which is to say, it might be stored both on my C: drive and on my removable drive.

And of course, my understanding of where the various date/times are all stored may be entirely incorrect. For example, RM and RM-To-Go could possibly be storing all the various date/times internally rather than relying on the Windows folder date/times. If so, I would still be curious to know where in the RM database and the various INI and XML files the date/times are stored so that I can look at them and try to figure out what's going wrong.

Jerry

[c24m48]

c24m48, on 18 February 2013 - 12:14 PM, said:

My computers are always kept in sync with atomic clocks on the Internet. If they differ, it would be by much less than a second.

In any case, it's hard to see how computers whose clocks were not rigidly accurate could be the problem. For example, suppose I run the RM-To-Go sync at exactly noon on Saturday to sync up my C: drive and my removable drive. At this point, my C: drive and my removable drive should both say that the last date/time modified was noon Saturday and the date/time of the last sync should also say noon Saturday. Then suppose I drive to my local public library where I would get on RM-To-Go for a couple of hours using one of the library's computers. And let's say that the library's computer is off by five minutes. So I finish at 3:00 but the library's computer says it's 3:05. When I get home, my database at home should say last modified at noon, the last date/time synced should still say noon, and my removable drive should say last modified at 3:05 instead of modified at 3;00 because of sloppy timekeeping at the library. But that shouldn't cause any problems with the next sync.

Well, there's another possibility. A computer's hardware clock should be set to Coordinated Universal Time (AKA Greenwich Mean Time) and then the computer's timezone offset should be set correctly. If the computer's hardware clock were to be set to local time and the timezone offset were set to zero, then I could see how there could possibly be a problem. That is, there could possibly be a problem if one of the computers involved with RM-To-Go were set up one way and and the other computer infovled with RM-To-Go were set up the other way. But my computers don't have this problem. And I would suspect most computers don't have the problem these days because Windows leads you in the direction of doing your timezones correctly. You tell it what timezone you are in and you tell it what the local time is, and then it figures out for you how to set the hardware clock to Coordinated Universal Time. It hides most of the complications from you so you don't have to deal with them.

Jerry

[TomH]

I don't think RootsMagic To-Go's Transfer Data system (note that it does not say "Sync") does anything more than compare file timestamps and sizes. I think a "conflict" is declared when the timestamps differ and the files are the same size (or maybe the older one is larger, too). A simple experiment:

• With both local and portable RM closed, transfer a database from the computer to the portable drive.
  
• Open RootsMagic To-Go - the files are declared to be the same. Close RM2go
  
• Open the local database with the local RootsMagic. Reopen RM2go. Now the local database is newer and same size. Close RM2Go.
  
• Open the database on the portable drive with the portable RM. Reopen RM2go. Now the local database is older, still same size and the files are in conflict!

So, without actually doing anything more than opening the database, the file timestamp is changed.

My conclusion is that you must not transfer from computer to portable while the computer's database file is open and you should never open it again until after you have done what you want with the portable version and have transferred from portable to computer. Otherwise, there is an ambiguity.

It would require a bunch more tests to flesh out all scenarios, e.g., what if the portable file has had deletions and is smaller than the older computer file?

[TomH]

I went a couple of steps further:

• With both RM's closed, transfer database from computer to removable. Open RM2go: same size, same time. Close RM2go.
  
• Open removable database and delete one person. Close database. Open RM2go: removable is newer, same size! No conflict. Close RM2go.
  
• Open removable database and Compact it. Close database. Open RM2go: removable is newer and smaller size. No conflict. Close RM2go.
  
• Open computer database and close it. Open RM2go: computer database is newer and larger size. Conflict! Close RM2go.
  
• Open computer database, Compact, close. Open RM2go: computer database is newer and still larger size. Conflict! Close RM2go.
  
• Open removable database and close it. Open RM2go: removable database is newer and still smaller size. Conflict! Close RM2go.
  
• Open computer database, delete same person, and close it. Open RM2go: computer database is newer and still larger size. Conflict! Close RM2go.
  
• Open computer database, Compact and close it. Open RM2go: computer database is newer and same size. Still conflict! Close RM2go.

Although the contents are identical and the file sizes are the same after step 8, there is still conflict. I think Jerry is right that there is another date other than the last modified date involved. Windows also keeps track of the date created:
November-‎10-‎12, ‏‎5:03:51 PM computer database file created
February-‎18-‎13, ‏‎10:25:25 PM computer database file modified
‎February-‎18-‎13, ‏‎9:59:52 PM removable database file created
‎February-‎18-‎13, ‏‎10:19:28 PM removable database file modified

I wonder if the Conflict flag is raised whenever the computer file modified timestamp is later than the creation timestamp for the removable.AND the removable's modified timestamp is later than its creation timestamp. That would flag that both files have been modified in some way. Unfortunately, merely opening the file changes the modified stamp!

[TomH]

Another observation about what RM2go gives the transferred database file:

• Created timestamp which is that of the system clock at the time of transfer.
  
• Modified timestamp which is the Modified timestamp of the source file with a possible discrepancy of 1 second or so.

I saw the discrepancy on the file transferred from computer to removable (latter +1s) but not in the reverse direction. The removable is formatted FAT32.

[c24m48]

I'm going to have to think about all of Tom's examples. I would only say that everytime I've put together a series of examples that seem to point to where the problem is, the examples ultimately are not repeatable.

In the meantime, here is one simple example. Start out with RM itself not running and run the RM-To-Go transfer utility to get everything synced up. Then on the same machine and without removing the USB memory device, start up RM as RM-To-Go on the USB memory device (RM itself is running from the memory device) and run all the file utilities in order, ending up with compacting the database on the USB memory device. Close down RM, and run the RM-To-Go transfer utility. In my simple test, it says to copy the data from the USB memory device (newer, smaller) to my C: drive (older, larger). So it's obviously going by newer/older rather than going by smaller/larger, and it will copy from smaller to larger without complaint. The glitch has to be a one or two second rounding error because of the limited precision of the date/time that is stored. It's clear why this would happen on a FAT32 USB memory device. The error will happen regularly on such a device. But it's not clear to me why the error will happen on the NTFS USB memory device that I am currently running.

That still begs the question in my mind about how the transfer utility *really* decides that there is a conflict. Is there a third date/time stored somewhere or not?

Jerry

[TomH]

Removable's timestamps:

1. Before opening and compacting:

• Created: February-‎18-‎13, ‏‎10:46:32 PM
  
• Modified: ‎February-‎18-‎13, ‏‎10:54:02 PM

2. After opening and compacting:

• Created: ‎February-‎18-‎13, ‏‎10:46:32 PM
  
• Modified: February-‎19-‎13, ‏‎8:18:52 AM

So Compacting does not change the Created date (the time of transfer) but does change the Modified date. If the one side's Created date is equal or later than the other's Modified date, then there is no conflict. If its Modified date and size is the same as the other's, it has not been changed. If its Modified date is later, then it is newer and as my earlier tests and yours indicated, size does not matter but is used as an indicator to the user to make a decision.

When you next get an unexpected Conflict, compare the two date properties for each of the two files. If we signify Created by C and Modified by M and subscripts 1 and 2 for the two files, there are four comparisons to be made between the two files:

• C1-C2
  
  
• M1-M2
  
  
• C1-M2
  
  
• M1-C2

Apart from timestamp errors, I am pretty confident that the behaviour of RM2go is dependent solely on these comparisons.

[c24m48]

TomH, on 19 February 2013 - 07:33 AM, said:

When you next get an unexpected Conflict, compare the two date properties for each of the two files. If we signify Created by C and Modified by M and subscripts 1 and 2 for the two files, there are four comparisons to be made between the two files:

• C1-C2
  
• M1-M2
  
• C1-M2
  
• M1-C2

Apart from timestamp errors, I am pretty confident that the behaviour of RM2go is dependent solely on these comparisons.

Will do, but I will have to wait for it to happen. I can't make it happen on command. Fortunately (or unfortunately ), I shouldn't have to wait very many days for the conflict to happen again.

And think I finally understand what you are saying about creation date/times. I think you are saying that what I have been calling the "third" date/time or the "sync" date/time is actually the creation date/time of the removable file. I've been so focused on the modified date/time of the removable file and the the modified date/time of the hard disk file that I have been neglecting to think about the creation date/time of either file.

Jerry

[c24m48]

TomH, on 18 February 2013 - 09:59 PM, said:

Another observation about what RM2go gives the transferred database file:

• Created timestamp which is that of the system clock at the time of transfer.
  
  
• Modified timestamp which is the Modified timestamp of the source file with a possible discrepancy of 1 second or so.

I saw the discrepancy on the file transferred from computer to removable (latter +1s) but not in the reverse direction. The removable is formatted FAT32.

I'm still looking at all of Tom's messages and thinking about them. If you take observation #1 and observation #2 from this particular message in combination,it means that the RM-To-Go transfer utility is creating a file that was modified before it was created. On its face,that sounds very strange, and it sounds like an impossible situation that is surely a bug. But the exact same thing happens if you use Windows copy/paste to make a copy of a file. And in a curious way, it all makes sense. The creation date/time of the copy is the date/time of the copy, and the modification date/time of the copy is the modification date/time of the original. There are lots of file and folder synchronization utilities in the world (some free, some not). Some of them use the creation date/tiome of the original file as the creation date/time of the copy, and some of them use the date/time of the copy operation as the creation date/time of the copy. I'm not sure which makes more sense. And I'm still not sure how this plays into the logic that the RM-To-Go transfer utility uses to detect that both the original and the copy have been modified.

The 1 second discrepancy that Tom saw on the modification date/time is almost certainly because FAT32 file systems store date/time stamps rounded off to the nearest 2 seconds. Again, I'm not sure how this plays into the logic that the RM-To-Go transfer utility uses to detect that both the original and the copy have been modified. I would love to say that the rounding to the nearest 2 seconds on FAT32 is probably the problem. But if that is the problem, then RM-To-Go would say that your files are still out of sync immediately after a sync operation, and RM-To-Go doesn't say anything like that.

Finally, now that I'm all fired up to look at Creation date/times and Modified date/times as Tom suggests, everything synced up just fine last night when I ran the RM-To-Go transfer utility. I will have to wait on another night (or two or three?) until the error recurrs.

Jerry

[c24m48]

I'm so excited! The error just happened, and I have some data to report. But it is very strange.

From outside RM (no RM software is running, neither RM itself nor the RM transfer utility)
Removable:	 Created: 2/20/2013 12:27:06 AM	 Modified: 2/20/2013 4:01:45 PM
Hard drive: Created: 2/20/2013 6:11:10 PM	 Modified: 2/21/2013 12:46:49 AM

These date/times are in accord with what actually happened. To wit,

• 2/20/2013 12:27:06 AM is when the RM-To-Go transfer utilitly copied from Hard Drive to Removable
  
• 2/20/2013 4:01:45 PM is when I closed RM on Removable
  
• 2/20/2013 6:11:10 PM is when the RM-To_go transfer utility copied from Removable to Hard Drive
  
• 2/21/2013 12:46:49 AM is when I closed RM on Hard Drive

So now I'm ready to run the transfer utility to copy from Hard Drive to Removable again, and all the date/times look perfect from outside of RM. The transfer utility does not display "Created" date/time, only "Modified" date/time. And it is showing 2/21/2013 12:46:49 AM as the "Modified" date time both for the Hard Drive and the Removable. I have not actually run the RM-To-Go transfer utility yet. I just started it up and got the conflict message before doing anything else. It seems like the RM-To-Go transfer utility at this point should still be showing 2/20/2013 4:01:45 PM as the "modified" date/time for the Removable. Maybe that's enough information for the development team to figure out what's going on.

Jerry

[c24m48]

One more piece of data about the error. After I do the manual override to tell the Rm-To-Go transfer utility to copy from Harddrive to Removable and before I actually click the Transfer button to make it go, it shows the correct "Modified" date/time for Removable as 2/20/2013 4:01:45 PM. That should further help the development team to identify where the bug is and fix it.

It's looking to me as if the bug has nothing to do with time zone glitches or esoteric and difficult calculations associated with the date/time formats used by Windows or the two second glitch on date/time stamps on FAT32 devices or anything like that. The error is looking much more mundane - like the transfer utility is sometimes picking up the "Modified" date/time from the Hard Drive and using it as the "Modified" date/time for the Removable device. But the bug is not repeatable. It happens sometimes and sometimes not, with no apparent rhyme or reason as to when it happens and when it doesn't.

Jerry

[c24m48]

Here's some new data. It feels like I'm running around in circles just a little bit because this new data would appear to get back to pointing towards some problem with a one or two second discrepancy in date/time calculations within the RM-To-Go transfer utility

On my main computer, I had everything all synced up between the Hard Drive and the Removable Drive and the RM-To-Go transfer utility said everything was fine. I started up the main RM and closed it down. I just opened my database and closed it and made no changes whatsoever. Then I ran the RM-To-Go transfer utility, and the conflict happened where it said I had updated both the Hard Drive copy of my database and the Removable Drive copy of my database (and of course I hadn't).

I noticed something a little different this time, and where I looked for problems this time may be a better place to look in general. Namely, I looked at the Modified date/times from outside of RM and campared them with the Modified date/times displayed by the RM-To-Go transfer utility. Things are off by a second or two as follows.

Modified date/time for Removable Drive

• 2/21/2013 4:26:19 PM as displayed from outside of RM
  
• 2/21/2013 4:26:20 PM as displayed by the RM-To-Go transfer utility

Modified date/time for Hard Drive

• 2/21/2013 5:17:20 PM as displayed from outside of RM
  
• 2/21/2013 5:17:22 PM as displayed by the RM-To-Go transfer utility

There are lots of folder and file synchronization products on the market, both free and not free. As I have mentioned before, all such products have to deal with one or two second discrepancies in date/times that are really equal. Such discrepancies are caused by the way Windows stores date/times. Such utilities usually deal with these discrepancies by treating plus or minus a second or two as being equal. However, the data above is really puzzling. That's because the discrepancies usually occur when comparing date/times between a Removable Disk and a Hard Disk. But this time, the discrepancies are between the interpretation of a single date/time value by the RM-To-Go transfer utility and the interpretation of the same date/time value from outside of RM.

All the date/times above are "correct" in the sense that the Removable Disk really was last modified about 4:26 and the Hard Disk really was last modified about 5:17.

Jerry