It happened to be Double Eleven some time ago. Usually before and after Double Eleven, there will be a small climax of part-time orders for Python. During this period, the demand for reptiles and data classification in various industries will increase dramatically!
Many friends in the circle are busy during weekends, and many of them earn tens of thousands these days.
Some orders taken recently (partial screenshots)
Which type of data is more profitable to develop and crawl?
Crawler, the most commonly used technology for taking orders;
Jobs that make you money quickly by taking more orders can be roughly divided into two categories:
-
Crawlers crawl data from websites or APPs, or provide crawlers directly to customers
-
Web back interface development, such as using Django to provide API interfaces
If you want to expand your order-taking business (including Web projects), in addition to crawlers, you also need to master at least one Python framework.
I recommend that Python beginners learn the Django framework. Django is a must-learn tool for novices to handle web projects!
Python-based web development technology stack
Regarding the Django framework, this is a complete web framework that helps you make many default choices. You can use these pre-included tool chains to build web services easily.
And thanks to Django’s well-received fully automated management backend, you only need to use ORM and simply define objects to automatically generate a database structure and a full-featured management backend.
Common technical issues
For programmers, making a living by taking orders is a good way, but if you plan to keep taking orders, you still need to consider your own situation.
Let me explain some points that need attention in advance;
01 Things to note when accepting orders
① When an order obviously cannot be fulfilled, the customer should be informed as early as possible. For example, it is obviously beyond the scope of one’s ability, or the demand cannot be fulfilled at all.
② It is best not to take more than three orders at the same time. Taking several orders at the same time will consume a lot of your energy.
③ When the customer’s requirements are vague, be sure to clarify all the requirements, otherwise you may continue to add details and requirements later. In addition, only by understanding the requirements can you give a suitable quotation.
④ Communicate with customers in a friendly manner. Don’t be anxious when you can’t get in touch. Just collect money and do the work without getting mixed up with all kinds of emotions. If the construction deadline is missed due to the inability to contact the customer, it’s not your fault.
④ After the cooperation, if you and the customer are satisfied with each other, you may wish to add a friend. Based on trust, you can contact us directly if you need it in the future.
02 Clarify the crawler process
If a novice wants to take orders, he must first understand how the crawler works;
A crawler usually consists of four steps: target information website, page crawling, page analysis, and data storage. The detailed process of crawling website resources is as follows:
① First import two libraries, used for requests and web page parsing respectively.
② Request the web page to obtain the source code
③ After initializing the soup object, use a browser to open the web page and locate the required resources.
④ Then analyze the source code of the location and find the tags and attributes used for positioning
⑤ Finally write the parsing code to obtain the desired resources
After thoroughly understanding the principles and steps, crawling data from ordinary websites is not a big problem, but it is far from enough to crawl data part-time.
Because the truly valuable resources are often found in large sites with complete anti-crawler measures!
03 Know how to deal with anti-crawling
Next, it involves how to deal with reverse crawling?
For example, common anti-crawler measures:
-
The target detects that the IP is blocked by a crawler
-
The target returned encrypted data
-
The target returned dirty and unrecognizable data
-
The target website must be logged in to access
-
Javascript dynamic rendering, crawlers cannot read
-
The target website has a verification code and cannot be accessed.
-
Ajax asynchronous transmission, the crawler captured empty information
-
Image camouflage and obfuscation + CSS offset + SVG mapping
At this time, you need to use the key technology of Python crawler – website anti-crawler strategy and its countermeasures.
If you fail to take the anti-crawler measures set up on your website, you will not be able to crawl valuable resources, let alone receive high-value orders!
What private jobs can I take?
Python crawlers mainly crawl data from websites, mini programs or APPs, analyze and process the data, or directly provide crawler programs and technical support to customers. This can be a side job.
When I first learned Python, a friend introduced me to a private job, which was to build a website for a company. I earned 3.5K from that job.
Since then, I have become more proficient, and in my spare time I have taken on many private jobs including data collection and processing, and website backend interface development. On average, I can earn about 5K a month from part-time private jobs.
01 Crawler service/project
After being introduced by acquaintances or taking orders through some outsourcing websites, after the content is developed, as long as no other problems occur, the basic monthly income of around 3-5K is not a problem.
Python is in greater demand for crawlers, website development, GUI interface or TK interface development, databases or some scripts, etc. These are relatively in demand.
For simple data capture, you can write the program in about ten minutes, and you can get two to three hundred directly, which is relatively easy.
Outsourcing unit price:
Taking ordinary crawler outsourcing as an example, it costs about 100-300. It is usually relatively simple. It can be judged according to the amount of data/task difficulty (requirement)/time. It can be completed in 30-60 minutes. Of course, the more skilled the technology, the faster!
Around 500-1000, it is a little difficult and can be done in 2-3 hours. The amount of data is large and the difficulty of the website may require an IP proxy.
Those above 1,000 involve anti-crawling, font encryption, JS decryption, APP data, etc. These are the most difficult parts of the crawler.
There are 2K/5k/10k/20k crawler orders. The higher the price, the longer it takes to do it. Some of them can’t even be completed by one person. At this time, you can also call other colleagues and friends to work together and set up an outsourcing studio. question.
02 Channel/Platform
Channels generally go through:
-
Order taking platform
-
Xianyu
-
Taobao
-
Q group to receive orders
-
WeChat order receiving group
(Q group and WeChat group are easy to provide source code without giving money. I won’t introduce too much about the others.)
Let me briefly share with you a few order-taking platforms:
-
Jiefang
-
Ape urgent delivery
-
code city
-
Programmer Inn
-
Developed by everyone
-
Zhu Bajie
-
Yipinweike
-
Open Source Crowdsourcing
-
Smart City Outsourcing Network
-
Realization network
-
Electric duck community
-
Remoteok
-
Toptal
-
AngelList
-
British selection
-
outsourcing network
03 Quantitative Trading Mining Information
It is very convenient to use Python to complete quantitative trading, but the content will be difficult for novices.
Quantitative trading is a data mining process. The difference is that you can use the available models to select stocks, time, etc. Fund management is the feature engineering we need to do.
04 Ghostwriting Program
Help customers ghostwrite the programs they need. Many e-commerce platforms are ghostwriters. You can get a LAMP management system for a few hundred yuan.
But this is basically the graduation project of college students, so you can open a store like this.
The above is shared here. If you are interested, you can try it yourself~
Writing crawler programs is a very useful skill point. The simplest purpose for adults to learn a new skill is two words – to make money!
If you need a complete set of python information, scan the QR code below to add it and get it
# Finally:
Python learning materials
If you want to learn Python to help you automate your office, or are preparing to learn Python or are currently learning it, you should be able to use the following and get it if you need it.
① A roadmap for learning Python in all directions, knowing what to learn in each direction ② More than 100 Python course videos, covering essential basics, crawlers and data analysis ③ More than 100 Python practical cases, learning is no longer just theory ④ Huawei’s exclusive Python comic tutorial, you can also learn it on your mobile phone ⑤Real Python interview questions from Internet companies over the years, very convenient for review
There are ways to get it at the end of the article
2. Python course video
When we watch videos and learn, we can’t just move our eyes and brain but not our hands. The more scientific learning method is to use them after understanding. At this time, hands-on projects are very suitable.
3. Python practical cases
Optical theory is useless. You must learn to follow along and practice it in order to apply what you have learned into practice. At this time, you can learn from some practical cases.
Four Python Comics Tutorial
Use easy-to-understand comics to teach you to learn Python, making it easier for you to remember and not boring.
5. Internet company interview questions
We must learn Python to find a high-paying job. The following interview questions are the latest interview materials from first-tier Internet companies such as Alibaba, Tencent, Byte, etc., and Alibaba bosses have given authoritative answers. After finishing this set I believe everyone can find a satisfactory job based on the interview information.
This complete version of the complete set of Python learning materials has been uploaded to CSDN. If friends need it, you can also scan the official QR code of csdn below or click on the WeChat card at the bottom of the homepage and article to get the method. [Guaranteed 100% free]