S
S
Stanislav2018-11-09 21:30:17
MongoDB
Stanislav, 2018-11-09 21:30:17

How to shorten update requests when counting document views?

I would like to reduce the number of calls to kong to update the document.
There are posts on the site when you go to the post, the stats.view +1 field is updated, there are about 50 such requests per second.
The selection, due to its load, does not occur in the usual way:
1. The very first request to the document collects all the necessary information from 3 collections, one of which is heavy and takes more than a second. After all the selections, it collects a large object with everything necessary and saves it to a separate mongo collection in a string field (object.toString()), let's call it cache
2. The second and subsequent requests go directly to the cache collectionif it contains the required information. Documents almost never change and the document cache can be stored for years
.

const [cache, count] = await Promise.all([getCache(request, response), updateStats(request.params)])
// updateStats - обновляет поле documen.stats.view+1

Ideally, I would like the following, to create some kind of container where to add the document ID and the number of views
[{
    _id: ObjectId("..."),
    count: 2000
},{
    _id: ObjectId("..."),
    count: 1000
}]

And once a day, by krone, update the necessary documents, which will avoid tens/hundreds of thousands of extra requests to the collection per day.
How can this be done, and is it possible? Do some kind of global container in the node?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
S
Stanislav, 2018-11-09
@ms-dred

In general, there is no alternative yet, I want to try the following method for unloading mongo
Instead of updating the collection, I decided to write identifiers in the views.log file

function updateStats(request) {
    return fs.appendFileSync("views.log", `${request.document}|`, 'UTF-8') // request.document = часть URL по которой идет выборка
}

Well, in fact, further along the crown I will pull the file, sort through the data, overwrite everything en masse at night
From the file pieces of code
//file.split('|').filter(function (e) { return e })

function setCountObject(request) {
    return documents = [], Promise.all(request.map(async (document) => {
        return (item = documents.find(e => e.url === document)), item && (item.count = item.count + 1) || documents.push({
            url: document,
            count: 1
        })
    })), documents
}

function updateDocuments(request) {
    return Promise.all(request.map(async (document) => {
        return Collection.updateOne({ url: document.url }, { $set: { "stats.view": document.count }})
    }))
}

Mongu is at the peak of traffic, of course, to unload this, you need to watch and test =)

V
Vitaly, 2019-06-15
@vshvydky

1. create a collection (table) of an id of something, a view counter
2. on a get request for obtaining data on this collection you hang up a middleware, in it you make a counter increment
3. give the client content
4. enjoy life
ps: about the counter increment

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question