D
D
Dmitry Filippov2018-12-07 00:09:39
Python
Dmitry Filippov, 2018-12-07 00:09:39

Why does she swear when sending data to MongoDB?

Hello.
I am writing an XML file parser in order to transfer all the data to the mongu.

def proccess_node():
  for event, element in etree.iterparse(osmfile, tag="node"):
    pool = {}
    tags_pool = []
    for child in element:
      if child.tag == "tag":
        tags_pool.append(child.attrib)
    pool['tags'] = tags_pool
    element.clear()
    insert_node(pool)

def insert_node(pool):
  client = MongoClient('mongodb://localhost:27017/')
  db = client.osm # db
  pp.pprint(pool)
  node_collection = db.Nodes
  node_id = node_collection.insert(pool).inserted_id

The process_node function collects the following dictionary:
{'tags': [{'k': 'name', 'v': 'Москва-Пассажирская'},
          {'k': 'railway', 'v': 'station'},
          {'k': 'uic_ref', 'v': '2000027'},
          {'k': 'alt_name', 'v': 'Москва-Октябрьская'},
          {'k': 'esr:user', 'v': '060073'},
          {'k': 'loc_name', 'v': 'Москва-Ленинградская'},
          {'k': 'name:esr', 'v': 'Москва-Пассажирская'},
          {'k': 'nat_name', 'v': 'Москва'},
          {'k': 'uic_name', 'v': 'Moskva Oktiabrskaia'},
          {'k': 'express:user', 'v': '2006004'},
          {'k': 'official_name', 'v': 'Москва-Пассажирская'},
          {'k': 'loc_name:website', 'v': 'http://www.tutu.ru/station.php?nnst=79310'},
          {'k': 'official_name:esr', 'v': 'Москва-Пассажирская'},
          {'k': 'official_name:website', 'v': 'http://pass.rzd.ru/timetable/public/ru?STRUCTURE_ID=5104&layer_id=5368&id=278&node_id=19'},
          {'k': 'official_name:express-3', 'v': 'Москва-Октябрьская'},
          {'k': 'official_name:esr:website', 'v': 'http://cargo.rzd.ru/cargostation/public/ru?STRUCTURE_ID=5101&layer_id=4829&page4821_2705=1&refererLayerId=4821&id=1090'},
          {'k': 'official_name:express-3:website', 'v': 'http://bilet.ru/rus/TrainDirectory.htm?firstsymb=%u041c'}]}

And then sends this dictionary to the database.
When running the code, the following error occurs:
bson.errors.InvalidDocument: Cannot encode object: {'k': 'name', 'v': 'Москва-Пассажирская'}

What could be the problem?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
R
Ruslan Gilfanov, 2018-12-07
@HorrorInferno

Try replacing nested dictionaries with SON objects:

from bson.son import SON

pool['tags'] = [SON([(k, d[k]) for k in d]) for d in pool['tags']]

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question