Tycoon Talk
Become a Big fish!
The number 1 forum for online business!
Post topics, ask questions, share your knowledge.
Tycoon Talk is part of Freelancer.com - find skilled workers online at a fraction of the cost.

PHP Forum


You are currently viewing our PHP Forum as a guest. Please register to participate.
Login



Freelance Jobs

Reply
Splitting a Large XML/KML File
Old 10-05-2007, 06:29 PM Splitting a Large XML/KML File
johnncyber's Avatar
Extreme Talker

Posts: 216
Trades: 0
I have a rather large (1.3 GB) KML file that I need to split into smaller more manageable files (that SimpleXML won't choke on). I have not been able to find a good script to do that, and I am not too familiar w/ XML in PHP. Has anybody had luck splitting up a large XML file into to smaller, well-formed XML files?
__________________
~Mark Romero
-Co Founder | Tech Guru,
Please login or register to view this content. Registration is FREE

-FireFox Advocate,
Please login or register to view this content. Registration is FREE

Last edited by johnncyber; 10-05-2007 at 10:25 PM..
johnncyber is offline
Reply With Quote
View Public Profile Visit johnncyber's homepage!
 
 
Register now for full access!
Old 10-05-2007, 06:45 PM Re: Splitting a Large XML/KML File
tripy's Avatar
Do not try this at home!

Posts: 3,621
Name: Thierry
Location: I'm the uber Spaminator !
Trades: 0
1.3 Gb...
Wow, that's a big Dom document...

I'm curious, but sorry to say I don't know of anything...
What about the structure though ?
Is it always the same pattern repeating over and over ?

If so, you should be able by parsing the document line by line to detect each primary entity beginning and end, and spit them out in smaller files.
__________________
Only a biker knows why a dog sticks his head out the window.
tripy is offline
Reply With Quote
View Public Profile Visit tripy's homepage!
 
Old 10-05-2007, 09:38 PM Re: Splitting a Large XML/KML File
johnncyber's Avatar
Extreme Talker

Posts: 216
Trades: 0
Quote:
Originally Posted by tripy View Post
1.3 Gb...
Wow, that's a big Dom document...

I'm curious, but sorry to say I don't know of anything...
What about the structure though ?
Is it always the same pattern repeating over and over ?

If so, you should be able by parsing the document line by line to detect each primary entity beginning and end, and spit them out in smaller files.
That would work considering it is a well formed XML document but I wanted to see if anybody had a solution before I went knee deep in code
__________________
~Mark Romero
-Co Founder | Tech Guru,
Please login or register to view this content. Registration is FREE

-FireFox Advocate,
Please login or register to view this content. Registration is FREE

Last edited by johnncyber; 10-05-2007 at 10:22 PM..
johnncyber is offline
Reply With Quote
View Public Profile Visit johnncyber's homepage!
 
Old 10-06-2007, 04:47 AM Re: Splitting a Large XML/KML File
ForrestCroce's Avatar
Half Man, Half Amazing

Posts: 3,023
Name: Forrest Croce
Location: Seattle, WA
Trades: 0
I think Tripy's approach will be the simplest to implement.
__________________

Please login or register to view this content. Registration is FREE
|
Please login or register to view this content. Registration is FREE
|
Please login or register to view this content. Registration is FREE
ForrestCroce is offline
Reply With Quote
View Public Profile Visit ForrestCroce's homepage!
 
Old 10-06-2007, 10:22 PM Re: Splitting a Large XML/KML File
johnncyber's Avatar
Extreme Talker

Posts: 216
Trades: 0
I ended up modifying a script I found on google (How-to Handle Large XML Files in PHP) to make it work. It basically follows the approach Tripy suggested. I ended up with 2500+ files, that were plenty small for SimpleXML to handle. Thanks for your help.

Note: The script I linked does NOT work out of the box, it takes some tweaking.
__________________
~Mark Romero
-Co Founder | Tech Guru,
Please login or register to view this content. Registration is FREE

-FireFox Advocate,
Please login or register to view this content. Registration is FREE
johnncyber is offline
Reply With Quote
View Public Profile Visit johnncyber's homepage!
 
Old 10-06-2007, 10:40 PM Re: Splitting a Large XML/KML File
Average Talker

Posts: 22
Name: Paul
Trades: 0
I did come across this...

http://www.hotscripts.com/jump.php?l...58&jump_type=1

Never used it before though, so couldn't say if it's any good. But it does split large files, and it also puts them back together again. Sounds a lot similar to WinRaR.
__________________
Free Web Hosting Available At Simply Web Host

Please login or register to view this content. Registration is FREE
smithster is offline
Reply With Quote
View Public Profile Visit smithster's homepage!
 
Old 10-07-2007, 02:21 AM Re: Splitting a Large XML/KML File
johnncyber's Avatar
Extreme Talker

Posts: 216
Trades: 0
Quote:
Originally Posted by smithster View Post
I did come across this...

http://www.hotscripts.com/jump.php?l...58&jump_type=1

Never used it before though, so couldn't say if it's any good. But it does split large files, and it also puts them back together again. Sounds a lot similar to WinRaR.
Unfortunately that would not have worked in this situation, because I needed to split it into well-formed XML documents that could be read using an XML parser.
__________________
~Mark Romero
-Co Founder | Tech Guru,
Please login or register to view this content. Registration is FREE

-FireFox Advocate,
Please login or register to view this content. Registration is FREE
johnncyber is offline
Reply With Quote
View Public Profile Visit johnncyber's homepage!
 
Reply     « Reply to Splitting a Large XML/KML File
 

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off





   
RSS Feed  Feeds: RSS   JS   XML
RSS Feed  Feeds for this forum: RSS   JS   XML



Page generated in 0.26263 seconds with 12 queries