Jump to content


Photo

Loooong Time to Create HTML pages


  • Please log in to reply
33 replies to this topic

#1 Jack

Jack

    Advanced Member

  • Members
  • PipPipPip
  • 834 posts

Posted 14 October 2013 - 04:10 AM

It takes 10+ hrs for RM 6.3.0.4 to create a tad over 5000 html pages from my 61KB RM file.

That file contains:

36736 people
11681 families
87517 events
5654 places
1552 sources
36383 citations
8050 multi-media items
8393 multi-media links

I have a relatively new computer with a fairly fast multi-core processor running Win7 in 64bit mode.

Creation begins rather fast with something close to 300 pages created within the first five minutes or so. Then, "things" begin to slow; barely creeping along as the number of pages grow.

At this rate, a 100KB file might take two days. :-)

Is this typical of what others are experiencing?
Jack

a posteriori

#2 kbens0n

kbens0n

    Advanced Member

  • Members
  • PipPipPip
  • 3388 posts

Posted 14 October 2013 - 07:22 AM

from my 61KB RM file.

That's a pretty amazing size for a database holding those items. When I create a empty database it defaults to 1306KB. Did you perhaps mean 61MB?

---
--- "GENEALOGY, n. An account of one's descent from an ancestor who did not particularly care to trace his own." - Ambrose Bierce
--- "The trouble ain't what people don't know, it's what they know that ain't so." - Josh Billings
---Ô¿Ô---
K e V i N


#3 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 7961 posts

Posted 14 October 2013 - 10:33 AM

I assume since you said HTML that you are creating the older HTML websites and not the new Publish Online one. I'm not sure if it is normal to take this long. If it is it's not something we will change because we are not updating the HTML websites anymore.

One thing you could do is turn off your virus scanning software to see if that is effecting the speed when generating the website.
Renee
RootsMagic

#4 Don Newcomb

Don Newcomb

    Advanced Member

  • Members
  • PipPipPip
  • 1008 posts

Posted 14 October 2013 - 10:55 AM

Yes, it takes a very long time to create a large HTML website. One thing that I do is open the Windows Task Manager, find the RootsMagic task and drop it's priority to "Low". This does not make it run faster but allows the computer to be used for other things while the website creation is going on. I've offered a suggestion that the program should do this automatically but last time I checked, I still had to do it manually.

#5 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 7961 posts

Posted 14 October 2013 - 12:33 PM

If the trigger to change the priority level in the Windows Task Manger would be controlled by the creating of the HTML websites then your not going to see that change in a future update to RM. You will need to continue to do that outside of the program. Like I said before we are not making any future changes to the HTML website area of RM.
Renee
RootsMagic

#6 Jack

Jack

    Advanced Member

  • Members
  • PipPipPip
  • 834 posts

Posted 14 October 2013 - 03:39 PM

That's a pretty amazing size for a database holding those items. When I create a empty database it defaults to 1306KB. Did you perhaps mean 61MB?


Yep, 61MB. :-(
Jack

a posteriori

#7 Jack

Jack

    Advanced Member

  • Members
  • PipPipPip
  • 834 posts

Posted 14 October 2013 - 03:43 PM

I assume since you said HTML that you are creating the older HTML websites and not the new Publish Online one. I'm not sure if it is normal to take this long. If it is it's not something we will change because we are not updating the HTML websites anymore.

One thing you could do is turn off your virus scanning software to see if that is effecting the speed when generating the website.


I haven't used or investigated the Publish Online utility yet. I publish my HTML pages to CD-ROM and not interested in publishing to the RM site.
Jack

a posteriori

#8 kbens0n

kbens0n

    Advanced Member

  • Members
  • PipPipPip
  • 3388 posts

Posted 14 October 2013 - 04:09 PM

I haven't used or investigated the Publish Online utility yet. I publish my HTML pages to CD-ROM and not interested in publishing to the RM site.


The Publish Online utility also saves it's files to your hard disk before offering to put a site up on RootsMagic.com for you, so you may want to try running it and perhaps you'll want to burn them instead.

---
--- "GENEALOGY, n. An account of one's descent from an ancestor who did not particularly care to trace his own." - Ambrose Bierce
--- "The trouble ain't what people don't know, it's what they know that ain't so." - Josh Billings
---Ô¿Ô---
K e V i N


#9 Jack

Jack

    Advanced Member

  • Members
  • PipPipPip
  • 834 posts

Posted 14 October 2013 - 06:27 PM

The Publish Online utility also saves it's files to your hard disk before offering to put a site up on RootsMagic.com for you, so you may want to try running it and perhaps you'll want to burn them instead.


I decided that I've give it a try and, after about 3.5 hrs., the marker is half-way across and hasn't moved in 2 of those.

I'll let it run overnight but don't think the program will tell me how long it took to complete the run.
Jack

a posteriori

#10 Don Newcomb

Don Newcomb

    Advanced Member

  • Members
  • PipPipPip
  • 1008 posts

Posted 14 October 2013 - 10:07 PM

The Publish Online utility also saves it's files to your hard disk before offering to put a site up on RootsMagic.com for you, so you may want to try running it and perhaps you'll want to burn them instead.

One little issue with the Publish Online website is that it is composed of thousands of tiny files. Each file is 1 to 6 kB but since on most current hard drive partitions the minimum block allocation is 4 kB the files are generally about 50% empty space. In other words, the database takes up twice as much space on your drive as is required to hold the data.

#11 kbens0n

kbens0n

    Advanced Member

  • Members
  • PipPipPip
  • 3388 posts

Posted 15 October 2013 - 01:00 AM

One little issue with the Publish Online website is that it is composed of thousands of tiny files. Each file is 1 to 6 kB but since on most current hard drive partitions the minimum block allocation is 4 kB the files are generally about 50% empty space. In other words, the database takes up twice as much space on your drive as is required to hold the data.


YES. PUBLISH results in 4-5 times more files than HTML and uses 3-4 times more storage space on disk or whatever media (even though total bytes actually used for files in PUBLISH is about 10% less than for HTML). Also, PUBLISH finishes a site creation in about one-third the time of HTML site creation. These are just my rough estimates. Fortunately, today's computers have an abundance of cheap storage (probably the least expensive parameter in overall system cost). Locally (on my computer), I just archive all the created website directories in .ZIP files and that reduces the disk space usage (a PUBLISH archive is just over 10% of original ondisk usage and HTML is just under 20% of original ondisk usage). I also archive media items and GEDCOMs. That doesn't help PUBLISH for online storage space usage, though ;-) This definitely limits RootsMagic.com hosting uploads, as well as users' personal webspace allocations. Hopefully, they'll switch to things like combining family data files at some higher-level abstraction (to avoid the large numbers of tiny files that waste disk clusters) and that includes turning individual website template graphics into a single graphic or several graphics and then using CSS Offset-style programming to position them. Likewise, integrating javascript and CSS files could also help. I'm sure they are aware of these shortcomings and are just trying to balance their implementation progress into the user curve of adopters (uploaders) to the rootsmagic.com PUBLISH site, while still dealing with the many other elements of program evolution/maintenance.

---
--- "GENEALOGY, n. An account of one's descent from an ancestor who did not particularly care to trace his own." - Ambrose Bierce
--- "The trouble ain't what people don't know, it's what they know that ain't so." - Josh Billings
---Ô¿Ô---
K e V i N


#12 Jack

Jack

    Advanced Member

  • Members
  • PipPipPip
  • 834 posts

Posted 15 October 2013 - 03:08 AM

It's a tad bit over 4AM here and PUBLISH is still running after nearly 12 hours. Marker is still half way across and hasn't moved overnight.

That's about 2 hours longer than HTML and seemingly not very efficient use of computer time.

The Magician may want to have a chat with whomever designed that utility or, perhaps, consider dumping it and working a bit more on the other website option.
Jack

a posteriori

#13 Don Newcomb

Don Newcomb

    Advanced Member

  • Members
  • PipPipPip
  • 1008 posts

Posted 15 October 2013 - 06:08 AM

This definitely limits RootsMagic.com hosting uploads, as well as users' personal webspace allocations. Hopefully, they'll switch to things like combining family data files at some higher-level abstraction (to avoid the large numbers of tiny files that waste disk clusters) .....

Of course, if you implement a web server and know in advance that it will be populated primarily with tiny files, that fact can be taken into account by picking a file system which deals with small files more gracefully than NTFS.

#14 Jack

Jack

    Advanced Member

  • Members
  • PipPipPip
  • 834 posts

Posted 15 October 2013 - 06:33 AM

Of course, if you implement a web server and know in advance that it will be populated primarily with tiny files, that fact can be taken into account by picking a file system which deals with small files more gracefully than NTFS.


I think that means I first have to know "stuff" about servers and NTFS that I don't yet know. :-)

I burn all of my HTML files to DVD-R and currently use about half or a little more when I include everything in the list in first entry to this thread.

Once created, it takes less than about 5 minutes to burn the DVD.

All I'm looking for is the ability to create the files in something less than half-to-a-full-day!
Jack

a posteriori

#15 Jack

Jack

    Advanced Member

  • Members
  • PipPipPip
  • 834 posts

Posted 15 October 2013 - 08:56 AM

OK...attempted to create a Publish Online file for a smaller database.

This one contains 24520 people
7685 families
46948 events
3020 places
801 sources
17000 citations
3052 multimedia items
2776 mutimedia links

It took about an hour to essentially create nothing!

RM created the following folders on my harddrive:

data
Images
javascript
stylesheet

Under data, it created folders: family, media, name, person, and files contacts.xml and settings.sml, both 1kb files.
Files in the media folder hold xml files that do not display and all sorts of medium, normal and tiny.jpg files that do display.
The name folder contains mostly 1kb *.xml files that do not display in my browser. Likewise the person folder.

The Images folder contains boiler plate images such as right and left arrows, etc. It also contains a folder named ie6, whatever that is and contains images beginning with "border."

The Javascript folder contains *.js files.

The Stylesheet contains *.css files.

Files in the family folder are mostly 1kb *.xml files that do not display in my Chrome browser.

It created the following files:

colorbox.css 6kb
groupsheet.html 2kb
help.html 3kb
index.html 2kb
individual.html 2 kb
names.html 2kb
pedigree.html 2kb

Except for the html files groupsheet, help, index, individual, names and pedigree, I find no other html files in any folder.

The index.html file contains the comments I entered as an introduction. That page also contains links to Index of Names, which is empty as well as an empty Pedigree Chart link.

I'm just clicking on the appropriate buttons as I navigate through the RM site making process. I have 931gb of free space on the hd, formatted to NTFS.

Either I'm doing something incorrectly or, the software is.

Does this implementation of a website utility rely on Java to function properly?

I'm beginning to feel a tad frustrated.
Jack

a posteriori

#16 TomH

TomH

    Advanced Member

  • Members
  • PipPipPip
  • 5876 posts

Posted 15 October 2013 - 09:12 AM

The Publish Online website makes heavy use of JavaScript. Depending on your browser and/or its settings, a local website may not display as a security measure. Somewhere in the Forum, there is a message describing what you have to do to get Chrome to show local. Iirc, Internet Explorer is okay, at least if security is set loose enough.

Edit: and the home page is index.html

Tom user of RM7550 FTM2017 Ancestry.ca FamilySearch.org FindMyPast.com
SQLite_Tools_For_Roots_Magic_in_PR_Celti wiki, exploiting the database in special ways >>> RMtrix-tiny.png app, a bundle of RootsMagic utilities.


#17 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 7961 posts

Posted 15 October 2013 - 09:21 AM

To get your local website to view using Chrome you need to close all Chrome windows first. After you have uploaded your files online then you have no problem viewing in Chrome.

If you want to judge the true speed of generating the websites you need to increase the priority level in the task manager. You've been changing the priority to low and that very much so can effect how long it's taking to create the files.
Renee
RootsMagic

#18 Jack

Jack

    Advanced Member

  • Members
  • PipPipPip
  • 834 posts

Posted 15 October 2013 - 09:31 AM

To get your local website to view using Chrome you need to close all Chrome windows first. After you have uploaded your files online then you have no problem viewing in Chrome.

If you want to judge the true speed of generating the websites you need to increase the priority level in the task manager. You've been changing the priority to low and that very much so can effect how long it's taking to create the files.


Are you telling me that I have to upload the files to a Roots Magic website BEFORE I can view them? If accurate, that's not good!

And, you might have me mistaken with someone else. I do not and have not changed any priorities in task manager.
Jack

a posteriori

#19 Renee Zamora

Renee Zamora

    Advanced Member

  • Support
  • PipPipPip
  • 7961 posts

Posted 15 October 2013 - 09:35 AM

Jack, you're right I had you confused with Don Newcomb.

You can view the Publish Online files without uploading them. You just need to be aware of the limitation using the Chrome browser.
Renee
RootsMagic

#20 Jack

Jack

    Advanced Member

  • Members
  • PipPipPip
  • 834 posts

Posted 15 October 2013 - 09:43 AM

Jack, you're right I had you confused with Don Newcomb.

You can view the Publish Online files without uploading them. You just need to be aware of the limitation using the Chrome browser.


Which are?

I'm presuming that I can dbl-clk on index.html to begin viewing?

Doing that has no effect even with Chrome completely shut down,

Ditto with IE10 as "default browser."

And, I find no option in RM to upload already created files without rebuilding.

Edit: Just wondering why I have to "jump through hoops" to view the files in a modern, up-to-date browser like Chrome!? Is it perhaps because the developer of this RM utility has not updated it to conform to the advancements made in browser creation? It doesn't even work in IE10.

I get the feeling that the RM pulls "stuff" out of the air, mixes it in with his own work and then lets it rot there, whether or not it works like users want it to; e.g., the county checker.

Sorry to be so blunt, but this needs to catch someone's attention. If I cannot successfully use added features, why do I need your software?
Jack

a posteriori