How to get web scraping orders and what skills to improve?

Question

Freelance

Hey toaster.
About two years ago, I asked my first question about what you can do freelancing in order to earn a little, literally for bread and butter.
Since then, I have significantly improved my English and some professional skills, but there is still no job.
So it's time to ask your second stupid question on the toaster. If I may, it will contain many small questions, which I will put in italics. Readers can answer who can.
I decided to specialize in web scraping: there are many orders, you can start small and gradually grow. Then you can place your spiders on virtual servers and at the same time understand administration. Then you can make a website aggregator to understand website building. Needless to say, I'm stuck on the first step.
There is a problem that is familiar to any newcomer working outside the team: any new task takes an excruciatingly long time to solve. When you know that a little advice can save many hours or even days. My first question is, are there any chat rooms or forums where scrapers can chat, where you can ask general questions like I'm asking here? Telegram, Slack, Skype, publics, forums. All of a sudden there is a party.
I'm not going to ask how to code. I posted only a couple of questions on SO, the rest of the answers I found myself or wrote bicycles.
Further. Is there anything other than upwork? Does it make sense to look at Russian-language exchanges, job sites, LinkedIn, advertise on Avito? On freelancer.com, 10-30 bids are collected per minute. You think, okay, I'm dumping. Look, among these 30 bids, there are also such smart people.
Very little work is published on other exchanges.
I don't like that upwork takes 20% - it's just a disgrace.
I don't like that they took my Rising Talent badge from me after I turned down the interview by selecting the "no time" option. At the same time, the profile was available. Don't care about this label, it demonstrates the attitude towards users. Could give a warning for the first time: they say, do not forget about the availability indicator.
Why the hell is there no filter by customer reputation when looking for a job? When looking for a freelancer, you can filter by job success. Another demonstration of the attitude of the exchange towards users.
I don't want to spend a huge amount of effort to get a good reputation on upwork and get banned. In my profile, I have a Russian address, and on the Internet, including on upwork, I go through vpn with a foreign ip. I just now realized that this could be a problem. This is problem?There is nothing about this in the User Agreement. What does it matter to them what my ip is? Googling about bans of this kind did not find anything.
Most importantly: no orders .
Can BeautifulSoup, lxml, Requests, Selenium, Scrapy. Already doing http requests in Scrapy. I do it for a long time, but it works. When there is no time, I take Selenium.
I have not figured out Pipelines yet, have not tried to save data to the database, have not worked with large amounts of data. ( Quick stupid question along the way: Do I need to install MySQL and MongoDB on my Windows or just a VM/server? )
more money. But actually, I decided to write this post now, when I realized that I have not been earning anything for half a year. A few penny orders, good reviews, and that's it. I could already earn something now, but the situation is this:

No one is even looking. Is my price tag too high? I don't want to work a few hours for 10 bucks. Even though they would not interfere, it is already quite shameful. Is the price tag too small? I'll try to increase for the experiment. Now it costs $8/hr, for works with a fixed payment I proceed from it.
Now I put a prettier photo, I found that I can already write a more interesting non-invented overview and did it, I changed the entry level to intermediate.
How and for what jobs to respond?
I was hired when I just wrote "Hello, I can help you" - on the advice of someone from the toaster.
I don't get hired when I have something real to write about a job.
When a person posts a very short description, I write: ready, I can, let's write the details. And he doesn't answer. Will something change if I copy-paste the cover letter with some water?
And closer to my specialization: I never even considered web research work. Is it worth paying attention? I'm afraid if you collect information from several sources, take the time to parse them all, the client will say: I already have all this in the database, let's do something else.
Thank you for attention.
UPD: You are all great.
Raised the bet. Two interviews this night, one of them even without my bid: the dude found me himself. But here the keywords in the profile played a role.
Face progress.

Answer 1

I wouldn't trust a person with an 8$/hour rate, try at least 15

Answer 2

The idea of ​​increasing earnings is simple:
Scraping tasks are very often simple typical ones.
You take on such a task - you add a couple of lines to your ready-made standard solution, and sell it to the client.
But it does not add qualification.
However, economically very nice.
If you are sure that you can solve the problem - your price tag should be at least $20 per hour.
At the same time, you can increase the hour by licking your already prepared solution.
But for the growth of qualifications, the task should not be typical.
However, you will be implementing it for a very long time.
It is not economically beneficial.
I wouldn't bet less than $15 an hour.
But here in an hour you have to work more.
Of course, I'm not talking about manual scraping.
I'm talking about automation. And the ability to work with this automation.

Answer 3

There are several requirements for scrapping:
1. speed of writing layout-resistant rules
2. auto-checking of such rules
3. multithreading
4. scrapper architecture for hosting conditions
Write any of your aggregators or service analytics, and show that you can really create scrapper systems: immediately people will be drawn to you.

Answer 4

Don't you think that you have too many complaints about the upwork? "I don't like one", "I don't like the other". It's not you who sets conditions for upwork, it's he who sets conditions for thousands of people like you.
At the same time, those thousands will not stumble on writing parsers. Why don't you like the 20% rate? Double your price tag and you will earn more, you have no right to look into the pocket of the exchange, they created a business and they earn on it. At the same time, they allow you to earn money with them. If you don't like their conditions - look for clients in other places, but will your competencies be enough for them if you get stuck on writing a parser along the way? Also, dumping at a rate of 8 bucks is great.

Answer 5

This chat is called google search There is a list of almost all exchanges
on upworkest.ru/spisok-frilans-birzh
It's probably idiocy to expect a job to find you when you don't know anything, put twenty exchanges a day first and you won't know when you do the job

Answer 6

Are you looking for job scraping, but you can't write such a job scraper for different exchanges yourself?
Suspicious.

Answer 7

When a person posts a very short description, I write: ready, I can, let's write the details. And he doesn't answer.

With a short vague description, see if the customer has experience and verification on the exchange. He himself may not understand what he wants, answer exorbitant requests for a penny.
Maybe you shouldn't waste your time on such applications.
There applications for the Internet in English, or in Spanish, Indian, etc. And you have experience in Runet.
Also Russian skills. writer in English exchange is not required.