Node.js implements ticket grabbing gadget & SMS notification reminder

Written in the preface

You must know that working in Shenzhen is very painful, especially in the Kexing Science and Technology Park where I work. There are so many people there. Going to work every day is like Spring Festival travel. If I can switch to work in Dachong, I will be happy. Unfortunately, it cannot be replaced.

Especially at my station, there are a lot of people waiting for the bus. The bus to work is so crowded that only a few people can squeeze on when the bus is full. Usually I only use two words to describe this kind of person: “bus monster”

I remember that my friend was as skinny as a monkey and could still get on the bus. I am 182cm tall and weigh 72kg. It is not a problem for me to squeeze into the bus. I can block it with my backhand and make a fortune silently. Auntie in front of you, please hurry up, auntie, don’t be so grindy, hurry up. Go up, auntie, huh? Do you still want to squeeze me out? Can you squeeze me out? You can squeeze me out! I was on the spot! Eat the car!

…

Ahem, it’s impossible to squeeze into a bus, because today I found an online bus booking official account [xxxxxx] that allows you to customize routes

However, tickets are often sold out, and I also found that sometimes someone refunds a ticket, and then there are free tickets. The key is that I can’t keep an eye on the official account all the time, so I wrote A small tool for grabbing tickets + SMS notifications

Get interface information

View page structure

This is the ticket booking page, which displays the ticket status of the current month. According to the diagram, red means full, green means purchased, and gray means unavailable.

If it is optional, it will be a small white square, and the remaining votes will be displayed below, as shown in the figure below:

We’re going to do this,

  1. Regularly capture the returned interface information
  2. Determine whether there are remaining votes based on the interface return value

Okay, let’s review the source code and look at the interface information. Wait, WeChat browser can’t review the source code, so

Use chrome to debug the WeChat public account web page

First of all, we face a problem. If we directly copy the URL of the official account webpage and open it in chrome, this screen will be displayed. It is redirected to this page by 302, so it will not work. You can only enter by obtaining OAuth2.0 authorization.

So we have to use the packet capture tool first to know what information needs to be brought when accessing the WeChat official account webpage on the mobile phone. At this time, we have to use the packet capture tool, because my computer is a Mac and I cannot use Fiddler, I use Charles vase, which is the man below

With the help of this tool, we can easily capture mobile phone data packets in just 3 steps:

  1. Get the local IP address and port
  2. Set up proxy for mobile Internet access
  3. Perform the above two steps in sequence

Get the local IP address and port

The first step is to find the port number. Generally, the default is 8088, but to confirm, you can open Proxy/Proxy Setting and take a look. Oh, it turns out that I set it to 8888 before.

Then find help/Local IP Address of Charles. Click it and you will see your local address. Find the local address and write it down. Then proceed to the next step

Set up proxy for mobile Internet access

First, make sure that the phone and the computer are connected to the same wifi, and then there will be proxy information in the wifi settings, such as my monkey meter… No, Xiaomi 9 phone! The settings are as follows:

Enter the previous step to obtain the host name and port number.

After completing the input, click OK. Charles will pop up a dialog box asking you if you agree to access the proxy. Just click Allow.

Visit the target webpage with your mobile phone

After we accessed the WeChat public account [xxxx] on our mobile phone and entered the ticket grabbing page, we found that Charles had successfully captured the web page information. When we entered the ticket grabbing page, he would initiate two requests. , one is to obtain document document content, and a post request is to obtain ticket information.

After careful analysis, I roughly understand the business logic:

The entire project technical station is java + jsp, traditional writing method, user authentication is mainly a cookie + session solution, and the front-end mainly uses jQuery.

When the user enters the page, it will carry query parameters, such as starting site, time, train number and other information and cookie request document, which is the circled area.

The core content we want: the calendar is not displayed at the beginning.

Because I have to ask again

In the second request, carry the cookie and the above query parameters to initiate a post request to obtain the ticket information for the current month, which is the calendar content.

The following is a request for ticket information for the current month, but it is found that it returns a bunch of html nodes.

Okay… I guess after getting it, append directly to div, and then render to generate the calendar content.

Then operate on the mobile phone, select two dates, then click to place an order, send a ticket purchase request, and pull the ticket purchase interface. Let’s take a look at the request and return content of the ticket purchase interface:

Take a look at the content of the request. Based on the meaning of the fields, you can roughly understand the route, time, ticket amount, and payment method.

Take a look at the returned content: Return a json string data, which probably covers the successful return code of the order, time, ID number and other information.

Record the required information

Based on the above analysis, let’s summarize the content: The user authentication of the entire project uses the cookie and session schemes, and the request data uses the form data method. , we all know the request fields, but one thing is that when requesting the remaining votes, the html node code is returned instead of the json data we expected. This is a problem, and we cannot understand it clearly at a glance. How is it displayed when there are remaining votes?

Therefore, we can only debug through chrome to find out how he determines the remaining votes.

Let’s find a notepad and record the information. The recorded contents are:

  1. Request the urladdress of the remaining ticket interface and ticket purchase interface
  2. cookieinformation
  3. Respective request parameter fields
  4. user-AgentInformation
  5. Respective response return content

Set up chrome

After having the above information, we can start debugging with chrome. First open More tools/Network conditions

Fill in user-Agent into Custom

Charles captures local requests

Because we need to fill the obtained cookie into chrome and access the web page as our user, we need to modify the cookie package when requesting the target address.

First we need to turn on macOS Proxy and capture our http request

Open chrome and access the target URL. We can see that Charles has captured the target URL address we visited, and then put a breakpoint on the target URL address to facilitate debugging.

Then visit again. At this time, the breakpoint will take effect. A tab named break points will pop up. You can see that the reason why we still cannot access the target URL is because of sessionId No, so we fill in the captured cookie and click execute

At this time, you can jump to the target page correctly.

I roughly took a look at its overall layout, and the jQuery code and CSS code, especially the calendar.

Examined the following elements and found:

  1. The structure of the small square is:
<td class="b">
<span>This is the date</span>
<span>If there are remaining votes, the number of remaining votes will be displayed</span>
</td>
  1. The style name of td is a which means it is not optional
  2. The style name e means full
  3. The style name d means purchased
  4. The style name b is what we are looking for, which means it is optional, that is, there are more votes

At this point, the entire ticket purchasing process is clear.

When we make a request through Node.js, we will process the returned data and use regular expressions to determine whether there are more votes than the class name b. If there are more votes, we can get the content of the remaining votes in the div and that’s it.

Node.js request target interface

Analyze the functional points that need to be developed

Before writing code, we need to think about the function points and what functions we need:

  1. Request remaining ticket interface
  2. Schedule request task
  3. If there are remaining tickets, it will automatically request the ticket purchase interface to place an order
  4. Calling Tencent Cloud SMS API interface to send SMS notifications
  5. Multiple users can grab tickets
  6. Grab tickets for a certain date

First mkdir ticket creates a folder named ticket, then cd ticket enters the folder npm init and it doesn’t matter if you press Enter a few times along the way. Let’s start installing the dependencies. Based on the above functional requirements, we probably need:

  1. The request tool depends on your personal habits. You can also use the native http.request. I choose to use axios here. After all, axios is The bottom layer of the node side also calls http.request
cnpm install axios --save
  1. Scheduled tasks node-schedule
cnpm install node-schedule --save
  1. Select the dom node tool cheerio on the node side
cnpm install cheerio --save
  1. Tencent text messaging dependency package qcloudsms_js
cnpm install qcloudsms_js
  1. Hot update package, Nodou’s mother, nodemon (actually you can use it without it)
cnpm install nodemon --save-dev

Develop the interface for requesting remaining tickets

Then touch index.js creates the core js file and starts coding:

First introduce all dependencies

const axios = require('axios')
const querystring = require("querystring"); //Serialize the object, you can use qs, it's the same
let QcloudSms = require("qcloudsms_js");
let cheerio = require('cheerio');
let schedule = require('node-schedule');

Then we first define the request parameters and come up with an obj

let obj = {
   <!-- -->
  data: {
   <!-- -->
    lineId: 111130, //Route id
    vehTime: 0722, //Departure time,
    startTime: 0751, //Estimated boarding time
    onStationId: 564492, //Scheduled site id
    offStationId: 17990,//arrival id
    onStationName: 'Baoan Transportation Bureau③', //Scheduled station name
    offStationName: "Shenzhen-Hong Kong Industry-Academic-Research Base",//Scheduled destination name
    tradePrice: 0,//Total amount
    saleDates: '17',//ticket date
    beginDate: '',//booking time, left blank, used to fill in the data after grabbing the remaining tickets
  },
  phoneNumber: 123123123, //User’s mobile phone number, the mobile phone number for receiving text messages
  cookie: 'JSESSIONID=TESTCOOKIE', // Captured cookie
  day: "17" //Set tickets for the 17th. This is mainly used to grab tickets for the specified date. If the tickets are not available, you can grab all the remaining tickets for the month.
}

Then declare a class named queryTicket. Why use a class? Because based on the fifth demand point, when multiple users compete for tickets, we new respectively That’s it,

At the same time, we hope to be able to record the number of requests for remaining tickets, and automatically stop the operation of querying remaining tickets after grabbing the tickets, so we add a counting variable times and a variable for whether to stop, Boolean valuestop

Write code: