B
B
brainflow2014-10-30 11:07:02
PHP
brainflow, 2014-10-30 11:07:02

What language to write XML parsing daemon in?

Prompt, in what language it is better to implement a demon for parsing huge XML.
I don't have many options, I know php, python.
Perhaps there is some solution with a not very high entry threshold, I would love to master new technologies, but, for example, C ++ will take, I believe, a lot of time.
How big uncles do it and on what, I also want it that way!
Thank you.

Answer the question

In order to leave comments, you need to log in

3 answer(s)
K
Konstantin Kitmanov, 2014-10-30
@brainflow

Huge XML in any language is parsed by streaming parsers like SAX .

U
ugodrus, 2014-10-30
@ugodrus

Demon - I think it's too strong a word. The demon still has to manage to feed a large file. Yes, and specify what to do with it. You probably use the standard xml library and, accordingly, a large file collapses the interpreter.
I highly recommend digging towards a homemade parser. As an example, PHP has an XML parser . With modest needs, it can parse at least 10Gb xml while consuming up to 10mb of memory. The main thing is that there are no time limits.
Implementation example here .
I made something similar for myself. But not as a full-fledged parser, but as an xml primary parsing utility for further processing outside of it and with support for some kind of queries (to do something with elements that have a specific node-path). Parsed a maximum of 100mb xml.

H
Halfi, 2014-11-06
@Halfi

If in php, then dig in the direction of XMLReader - an xml parser built on SAX and reads the xml file not all at once, but from node to node.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question