Menu
▾
▴
#8 memory error with large log files
I got an memory error by applying the google sitemap
generator 1.4 to my server logfile with more than
800000 lines.
Solution:
I applied a solution to this problem proposed here:
http://www.thegillis.net/programming_papers/python/
google_sitemap_large_log_file_fix.html
I worked as expected.
This method should be integrated in the sitemap
generator.
Thorsten
Logged In: YES
user_id=695210
URL is dead, proper url can be found here:
http://www.thegillis.net/index.php?option=com_content&task=view&id=38&Itemid=31
Thanks for your patch, I'm using it on v1.4 :-)
I have fixed sitemap_gen.py so it now does not consume enormous amount of memory and python does not die with MemoryError. Look here http://www.bashkirtsev.com/2009/05/14/sitemap/