Jump to content


Photo

Storage of Media Files


  • Please log in to reply
83 replies to this topic

#61 Rick Landrum

Rick Landrum

    Advanced Member

  • Members
  • PipPipPip
  • 381 posts

Posted 10 March 2019 - 01:19 PM

 

Rick, no telling if my answer will be satisfying, but here goes.  I never use Tree Share to bring down media items, but rather download separately and manually link.  I do the same with Family Search, Find My Past, and all the rest of the paid repositories.  I separately maintain my Ancestry tree, and ne'er the programs shall meet!  Lots of extra work, to be sure, but I'm insulated from programming decisions made by Ancestry and indeed, Roots Magic.  My manual implementation of repository connection ensures data and filing integrity and avoids the problem you've mentioned of finding and renaming files brought in by the various APIs.

-------

 

The reasons for any particular subfolder structure (indeed, even having one at all!) lie with the desires and processes of the individual family historian.  Use what works for you, not only in filing, but also in source lumping, citation format and every other aspect of this fantastic pursuit!  Hope this helps, I'd be delighted to explain further.

Jim,

I like your idea of having a separate sub-folder for each person, or non-person specific topics such as cemeteries etc. This would make searching more simple. (by the way - while testing today I noticed that RM will handle movement to a sub-folder without requiring a re-link to the media.) I have been using Tree Share to bring in items to RM from Ancestry, but it usually causes so much trouble with having to rename and relink that I might as well start saving my own media and do it right to start with. I use a person centric syntax for naming media (Last, First, Middle, (Maiden) - (bxxxx) - Description). I also have a list of standard descriptions (birth, death, marriage, military, etc.). These work well, and I think the addition of subfolders will also help.The more I can do to keep this simple and fast, the better.

Thanks for your input.

Rick


RickL


#62 Rick Landrum

Rick Landrum

    Advanced Member

  • Members
  • PipPipPip
  • 381 posts

Posted 10 March 2019 - 01:55 PM

Make sure you backup your media files and structure and your database before embarking down this path.

Tom,

Quick question .......

While working with RM today I started getting a popup error message that says

"SQLite Error 11 - database disk image is malformed"

I have not run an SQL script in some time and I had no problems previously after running it.

When I try to open my Media Gallery, I get this message. I kill it and then try to run "Fix Broken Media Links" and it comes up several times. I kill it each time and eventually the relink will complete and appears to work OK. 

Any idea what this is about? How to fix?

Thanks 

Rick


RickL


#63 zhangrau

zhangrau

    Advanced Member

  • Members
  • PipPipPip
  • 1518 posts

Posted 10 March 2019 - 03:31 PM

I am neither an SQL jockey nor a database expert, just an experienced user of RM.

 

Have you tried running the Database Tools?

 

Immediately after opening the database, run the first three tools (Check integrity, Rebuild indexes, Remove phantoms).

Close and re-open the database and run the fourth tool (Compact).



#64 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3569 posts

Posted 10 March 2019 - 04:28 PM

I could put all my RM media files in one humongous folder and everything would just work. Which is to say my file naming conventions would not cause any collisions between file names. But for me personally it still works better to break things down into sub-folders. My sub-folder structure makes searches faster and it helps me think better about my data. Or maybe I'm just a simple country boy and like things in smaller pieces.

 

My sub-folder structure is not people oriented. I tried people oriented and it didn't work for me. I tried lots of other things that didn't work for me. What finally worked for me was the realization that I needed to have my own source templates in RM - one source template for census records, one source template for birth records, one source template for death records, one source template for marriage records, etc. Then I needed to store my media files using exactly the same system - one sub-folder for census media, one sub-folder for birth media, one sub-folder for death media, one sub-folder for marriage media, etc.

 

An exception is census media. I further subdivide it into additional sub-folders that are broken out first by state and then I further subdivide it into into  county.

 

The naming conventions for most of my media files is somewhat people oriented - death media for John Doe includes John Doe's name and the string "death" stored in the death sub-folder, birth media for John Doe includes John Doe's name and the string "birth" stored in the birth sub-folder, etc.

 

Census is again a bit of an exception because  multiple families that I research are often found on the same census page. So file names for census include the year, state, county, and page number - but no people names. I just don't think it makes sense to do so for census images.

 

Jerry



#65 Rick Landrum

Rick Landrum

    Advanced Member

  • Members
  • PipPipPip
  • 381 posts

Posted 10 March 2019 - 07:47 PM

Tom,

Quick question .......

While working with RM today I started getting a popup error message that says

"SQLite Error 11 - database disk image is malformed"

I have not run an SQL script in some time and I had no problems previously after running it.

When I try to open my Media Gallery, I get this message. I kill it and then try to run "Fix Broken Media Links" and it comes up several times. I kill it each time and eventually the relink will complete and appears to work OK. 

Any idea what this is about? How to fix?

Thanks 

Rick

After posting this question, I rebuilt my media directory by moving all my files to a new one that I gave a new name. I then ran the RM fix broken media links. Next I ran all the RM database tools. Everything is working fine now. Must have been some corruption in my media directory.

Thank you all for responding.

Rick


RickL


#66 Rick Landrum

Rick Landrum

    Advanced Member

  • Members
  • PipPipPip
  • 381 posts

Posted 11 March 2019 - 09:04 AM

I could put all my RM media files in one humongous folder and everything would just work. Which is to say my file naming conventions would not cause any collisions between file names. But for me personally it still works better to break things down into sub-folders. My sub-folder structure makes searches faster and it helps me think better about my data. Or maybe I'm just a simple country boy and like things in smaller pieces.

 

...................................

Jerry,

I agree with all you said, but for me the person centric folders seem to work the best. Same idea, just a different twist. I'm also a big believer in the "KISS" approach (keep it simple stupid). That always works best for me.  :)

Thanks

Rick


RickL


#67 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3407 posts

Posted 11 March 2019 - 11:50 AM

Jerry,

I agree with all you said, but for me the person centric folders seem to work the best. Same idea, just a different twist. I'm also a big believer in the "KISS" approach (keep it simple stupid). That always works best for me.  :)

Thanks

Rick

 

If your file naming system is robust and distinct enough you can change any folder structure at any time without detriment.


We are all limited by our visions and abilities

Whilst we can borrow from the visions of others we cannot always deliver.

 

User of Family Historian 6.2.7, Rootsmagic 7.6.0, Family Tree Maker 2014 & Legacy 7.5

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#68 thawkes

thawkes

    Member

  • Members
  • PipPip
  • 9 posts

Posted 15 March 2019 - 10:55 AM

Alright, time for me to step into this thread and add my thoughts.  Just like cj1260 said previously, I like to organize my media by person.
 
In the example below Reginald Adrian, and Janet Hawkes are married.
 
Typical Folder Structure:

  • Adrian, Reginald
  • Hawkes, Janet
  • Hawkes, Thomas
  • Walker, George

 
Inside each folder are pictures organized like this:

  • Adrian, Reginald family.tiff
  • Adrian, Reginald family(annotated).xcf
  • Adrian, Reginald family.png (small)
  • Adrian, Reginald family.jpg (thumbnail)
  • Adrian, Reginald portrait.tiff
  • Adrian, Reginald portrait.png (small)
  • Adrian, Reginald portrait.jpg (thumbnail)

The tiff files are 1200 DPI images that I have scanned from family photos and saved into an uncompressed tiff image.  This is my method of futureproofing these images as much as I can.  You know the images you archived 20 years ago look simple and grainy compared to the technology now.  And the technology in a further 20 years will make our methods look simplistic.  Anyways...

 

The "XCF" file is a gimp filetype / adobe photo shop (whatever you use is fine) where I have annotated the family image from left to right, so generations in the future can tell who is who.  They wont have my grandmother around to tell them that Reginald is on the left, and Thomas is on the right.  I've downloaded an old fashioned "rough_typewriter" font for the annotation to make it look old.

AU7kGnn.png

 

Then I resize the image down to a high quality png image that is 2000 pixels wide.  Saved with a compression ratio of "6". I call this my small image.

 

Then I further resize the image down to 500 pixels wide and save it into a lossy jpg image at 90% quality and call that my thumbnail.

 

However the pictures for Janet Hawkes are saved in a different folder because she has a different birth name.  I didn't see the point in merging husband and wife into one folder.  But that leads me to a difficult question.  What do I do with the "family" pictures, and the census documentaion that have multiple people from a family?  Do I make a copy for each folder and double the size of my genealogy folders? Do I make shortcuts around my folders and store the original family photo in Reginalds folder?

 

I guess, as I go forward I find that I have more and more items to add into these folders like: census, birth certificate, marriage certificate, hospital records, headstones, each of which I guess I can wrap in square brackets to describe what it is.

  • Adrian, Reginald [1911-census]
  • Adrian, Reginald [1926-census]
  • Adrian, Reginald [birth certificate]
  • Adrian, Reginald [marriage certificate]
  • Adrian, Reginald [hospital records]
  • Adrian, Reginald [headstone]
  • Adrian, Reginald [death certificate]
  • Adrian, Reginald [portrait]
  • etc

And I'll have to start using photo metadata to allow me to search though my genealogy folders quicker.

 



#69 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3569 posts

Posted 15 March 2019 - 04:54 PM

Both census records and marriage/divorce type records argue *against* organizing by person.

 

There are additional similar issues. I have a lot of city directory records, and they are similar to census in that many people can be on the same page. Obituaries mention lots of people. I suppose obituaries could be organized by person, by the deceased. Wills and settlements mention lots of people. I suppose wills and settlements could be organized by person, by the deceased. Deeds have at least a seller and a buyer, and often several of both. Court cases often mention lots of people, and usually have an X vs. Y element to them - so organize by person X or by person Y.

 

Jerry



#70 KFN

KFN

    Advanced Member

  • Members
  • PipPipPip
  • 212 posts

Posted 15 March 2019 - 08:23 PM

Both census records and marriage/divorce type records argue *against* organizing by person.
 
There are additional similar issues. I have a lot of city directory records, and they are similar to census in that many people can be on the same page. Obituaries mention lots of people. I suppose obituaries could be organized by person, by the deceased. Wills and settlements mention lots of people. I suppose wills and settlements could be organized by person, by the deceased. Deeds have at least a seller and a buyer, and often several of both. Court cases often mention lots of people, and usually have an X vs. Y element to them - so organize by person X or by person Y.
 
Jerry

. This is the primary reason I don’t organize by person. Most sources could contain information about multiple people so I only organize by source type.

#71 Vyger

Vyger

    Advanced Member

  • Members
  • PipPipPip
  • 3407 posts

Posted 16 March 2019 - 07:32 AM

Alright, time for me to step into this thread and add my thoughts.  Just like cj1260 said previously, I like to organize my media by person.

 

I have probably said it before but at fear of repeating myself and for what is is worth I organise by main family name then record type with few exceptions and I believe my file naming system stands on it's own merits, so

 

Genealogy

--Eagleson

----Births

----Deaths

 

using some examples from above and assuming Adrian is the surname my filenaming system would be; 

  • Adrian, Reginald [1911-census] - becomes adrian-reginald,1911-anytown-anystate-(cens)
  • Adrian, Reginald [birth certificate] - becomes adrian-reginald,1901-jan-01-(birt)
  • Adrian, Reginald [marriage certificate]- becomes adrian-reginald,doe-jane,1910-feb-01-(marr)
  • Adrian, Reginald [headstone] - becomes adrian-reginald,1920-mar-01-anycemetery-(buri)
  • etc

I know the comma is not a recommended file name character but I can easily make global file name changes via scripts in Rootsmagic and Windows as the file name structure is known and rigid. I can also build gedcom files from correctly named media collections on my hard drive with media linked to the correct event type and then work furtuer in Rootsmagic to beef things up is that course of action looks more productive.

 

Aside from folder assignments the file naming system is the one most important aspect, for example I have 54 John Graceys on the 1911 census so the file naming system thawkes outlined would not be distinct enough for my uses.


We are all limited by our visions and abilities

Whilst we can borrow from the visions of others we cannot always deliver.

 

User of Family Historian 6.2.7, Rootsmagic 7.6.0, Family Tree Maker 2014 & Legacy 7.5

 

Excel to Gedcom conversion - simple getting started tutorials here

 

Root


#72 Rick Landrum

Rick Landrum

    Advanced Member

  • Members
  • PipPipPip
  • 381 posts

Posted 16 March 2019 - 11:01 AM

I am following this discussion with great interest. There have been numerous good suggestions and ideas presented. (....wish I had given my naming conventions a lot more thought when I began using RM.....). While changing folder names and then moving media files between subfolders and relinking within RM is relatively easy to do, changing file names and relinking them to RM is a different problem. I currently have tens of thousands of media files, and the thought of renaming them to conform to new (although better) naming templates is daunting. As far as I know, the only process for doing this is "one by one". For now, I am concentrating on changing folder names, but I am only using new file naming templates on new additions to my tree, at least until I find a way to do mass updates and relinking. (the exception would be any review/revision of existing persons in my tree, I am changing those as I go).
 
Any suggestions regarding how to make these changes, and relinking to RM, in mass would be greatly appreciated.
 
Thanks
Rick

RickL


#73 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 6252 posts

Posted 16 March 2019 - 06:35 PM

Mass renaming and reorganising using a structure involving source type, or event type or name of person has to be derived from the database. Windows commands have to be generated to move and rename files while the former paths and filenames recorded in the database must be updated to the new. What I demonstrated in the SQLite script to rename cryptic filenames for TreeShared Ancestry Sources media files could be extended to other media and to moving files.

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#74 Rick Landrum

Rick Landrum

    Advanced Member

  • Members
  • PipPipPip
  • 381 posts

Posted 17 March 2019 - 10:54 AM

Mass renaming and reorganising using a structure involving source type, or event type or name of person has to be derived from the database. Windows commands have to be generated to move and rename files while the former paths and filenames recorded in the database must be updated to the new. What I demonstrated in the SQLite script to rename cryptic filenames for TreeShared Ancestry Sources media files could be extended to other media and to moving files.

Thanks Tom,

I suspected as much. For now I am going forward with a new (better) folder structure, and new (better) media naming templates (for new adds to database). Renamiing my old media files will have to wait until I'm more comfortable with an automated process. I'm still learning SQL so I'll experiment with the SQLite script (on backed up copies of my data base and media directory).  In the meantime, the old and new ways of naming & saving media can coexist with no problems other than making searches a little more difficult.

Appreciate it.

Rick


RickL


#75 RDHansen

RDHansen

    Member

  • Members
  • PipPip
  • 19 posts

Posted 02 June 2019 - 03:20 PM

Media file linking question. My media files are currently linked to my RootsMagic Media File location. How do I un-link a media item from there without deleting it all together? I then would like to use the re-linking tool to link the media item located in my main genealogy database location. RD Hansen

#76 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3569 posts

Posted 02 June 2019 - 04:04 PM

Media file linking question. My media files are currently linked to my RootsMagic Media File location. How do I un-link a media item from there without deleting it all together? I then would like to use the re-linking tool to link the media item located in my main genealogy database location. RD Hansen

 

On a case by case basis. you can go into Media Gallery, highlight an item, Properties, Change media file, and then navigate to and select the same named (or different named!) file in a different folder.

 

If you have many files in one folder and all the same files in a different folder, you can Search, Search and Replace, Media File names. At that point you can replace string xxx with string yyy (whatever those strings are) in all the file names in your RM database - or in your case in all the file paths to change which folder RM uses to reference the files.

 

The latter approach is so powerful that it can be a little dangerous if you are not careful. You might want to practice in a copy of your database to be sure you are getting the results you need.

 

Jerry



#77 RDHansen

RDHansen

    Member

  • Members
  • PipPip
  • 19 posts

Posted 02 June 2019 - 06:15 PM

Thanks Jerry, your case by case method worked perfectly. I will leave your other “dangerous” method alone. Appreciate your assistance. RD Hansen

#78 Rooty

Rooty

    Advanced Member

  • Members
  • PipPipPip
  • 164 posts

Posted 05 June 2019 - 09:42 AM

useful thread. I did not realize that backing up with media destroyed any media folder organization into subfolders. This makes that option useless.

 

Unless you organize the media folder with subfolders you cannot reasonably check for missing, duplicate or misplaced items. 

 

Many novice computer users seem to not grasp basic file, folder, data, backup and program concepts (to their peril). An hour with a simple user book like the "teach yourself visually" series would be so easy.



#79 Jerry Bryan

Jerry Bryan

    Advanced Member

  • Members
  • PipPipPip
  • 3569 posts

Posted 05 June 2019 - 01:40 PM

useful thread. I did not realize that backing up with media destroyed any media folder organization into subfolders. This makes that option useless.

 

 

A minor clarification: just doing a backup with media doesn't destroy media folder organization in your production files. It's only upon a restore that your media folder organization is destroyed in your production files. But the combination of all media files into a single folder does take place during the backup process when the media files are being written into the backup file. The restore only mimics what it finds in the backup file.

 

Jerry



#80 kbens0n

kbens0n

    Advanced Member

  • Members
  • PipPipPip
  • 3458 posts

Posted 05 June 2019 - 02:25 PM

FURTHER clarification:

During a Backup (with media):
- RootsMagic archives a copy of the database and a single folder containing all database-linked media files within the .RMGB file.
- The original database and media files remain unchanged.
- If you choose to restore that created backup AND do not select to include the backed-up media files... RootsMagic restores ONLY the database AND preserves the links to all original database media files.
- If (and ONLY IF) you choose to restore the created backup AND select to include the backed-up media files... RootsMagic changes the links within the database to point to that single Media folder for its media files.

TIP:
NEVER restore a backup to the database's original file location, initially! Choose a different folder location. In the event that the backup did not successfully capture all your files correctly, any attempts at overwriting may result in the inability to recall or rescue your data afterwards. Once you're sure you have a restorable database, you can elect to do a separate File->Restore to overwrite the original location, if you so choose.

---
--- "GENEALOGY, n. An account of one's descent from an ancestor who did not particularly care to trace his own." - Ambrose Bierce
--- "The trouble ain't what people don't know, it's what they know that ain't so." - Josh Billings
---Ô¿Ô---
K e V i N