TransBigData is a Python package developed for traffic spatiotemporal big data processing, analysis, and visualization. This article will use it to implement personal mobile phone positioning analysis
But in fact, traffic spatio-temporal big data is not limited to the data generated by transportation vehicles. A large amount of data is also generated in our daily lives. For example, our mobile phones record the places we have visited; when using city bus IC cards, shared bicycles and other services, the service provider can know the time and location of these travel needs, etc.
Introduction to TransBigData
TransBigData is a Python package developed for traffic spatio-temporal big data processing, analysis and visualization. TransBigData provides a fast and concise method for processing common traffic spatio-temporal big data (such as taxi GPS data, shared bicycle data, bus GPS data, etc.).
Currently, TransBigData mainly provides the following methods:
(1) Data preprocessing: Provides methods for quickly calculating basic information such as data volume, time period, sampling interval, etc. for data sets. It also provides corresponding cleaning methods for various data noises.
(2) Data rasterization: Provides a method system for quickly learning Python (rectangular, triangular, hexagonal and geohash rasters) to generate and match multiple types of geographical rasters within the study area, which can be quickly vectorized The algorithm maps spatial point data onto a geographic raster.
(3) Data visualization: Based on the visualization package keplergl, data can be displayed interactively and visually on Jupyter Notebook with simple code.
(4) Trajectory processing: Generate trajectory line types from trajectory data GPS points, densify and sparse trajectory points, etc.
(5) Map base map, coordinate conversion and calculation: Load and display the coordinate conversion between the map base map and various special coordinate systems.
(6) Specific processing methods: Provide corresponding processing methods for various types of specific data, such as extracting order starting and ending points from taxi GPS data, identifying residence and work place from mobile phone signaling data, and constructing network topology from subway network GIS data structure and calculate the shortest path, etc.
TransBigData can be installed through pip or conda. Run the following code in the command prompt to install:
pip install -U transbigdata
After the installation is complete, run the following code in Python to import the TransBigData package.
import transbigdata as tbd
Mobile phone signaling data reading
Mobile phone signaling data refers to the information exchanged between mobile phones and communication base stations, including location, communication duration, communication frequency and other data. These data can be used to analyze users’ travel behavior, living habits, etc., and can also be used in urban traffic management, commercial marketing and other fields.
Using the Python open source library TransBigData, you can quickly and efficiently process, analyze, and mine mobile phone signaling data, identify information such as travel and stay, residence and work place, and draw activity diagrams for easy analysis.
Principle of collecting mobile phone signaling data
First, we will use Python’s pandas library to read the data. Pandas is a powerful data processing library that provides flexible data structures and data analysis tools that can easily operate and analyze various data. We will use Pandas to read a CSV file containing mobile phone signaling data and store it in a Pandas data frame.
We need to convert the time field into the correct format for subsequent data processing. We use Pandas’ to_datetime function to convert the time field to datetime format. Then, we sort the data in chronological order for subsequent data processing:
import pandas as pd import transbigdata as tbd data = pd.read_csv(r'data/mobiledata_sample.csv') #Ensure that the time column is accurately identified (very important) data['stime'] = pd.to_datetime(data['stime'], format='%Y%m%d%H%M') data = data.sort_values(by = ['user_id','stime']) data.head()
The results are shown below.
▲ Overview of mobile phone signaling data
Identify trips and stays
Identifying trips and stays is an important step when processing mobile phone data. Based on mobile phone recognition of travel and activities, path analysis, travel mode analysis, crowd analysis and other work can be further carried out.
Activity: Mobile phone data can construct individual travel chain information by continuously tracking individual travel trajectories. Generally speaking, if a mobile phone user stays in a certain location for more than 30 minutes, we can consider that the user has activities here.
Travel: If two consecutive activities generated by a user occur in different geographical locations, the user can be considered to have traveled. The starting point of the trip is the geographical location of the previous activity in two consecutive activities. The start time of the trip is the end time of the previous activity. The end point of the trip is the geographical location of the later activity. The end time of the trip is the start of the latter activity. time. In short, the user’s movement between activity points is regarded as the user’s trip.
Activity and travel identification ideas
Using the mobile phone signaling data processing method provided by TransBigData, you can first map the data to a raster, and treat the data in the same raster as being at the same location to avoid data positioning errors that cause the same location to be recognized as multiple. Then, trips and stays can be identified from the mobile phone data using the tbd.mobile_stay_move function:
#Get raster parameters params = tbd.area_to_params([121.860, 29.295, 121.862, 29.301], accuracy=500) #Identify trips and stays from mobile phone data stay,move = tbd.mobile_stay_move(data,params,col = ['user_id','stime','longitude', 'latitude']) The result is as follows.
Stay recognition results
▲ Travel recognition results
Identify place of residence and place of work
Identifying users’ job and residence information through mobile communication data is one of the basic tasks of research. In TransBigData, based on the stay activity point, the place of residence can be identified using the tbd.mobile_identify_home method, and the place of work can be identified using the tbd.mobile_identify_work method. The specific rules are:
The residence identification rule is the place where you stay the longest during the night time period.
The work place identification rule is the place where you stay the longest during the day on weekdays (the average daily duration is greater than minhour).
The specific usage is as follows:
#Identify place of residence home = tbd.mobile_identify_home(stay, col=['user_id','stime', 'etime','LONCOL', 'LATCOL','lon','lat'], start_hour=8, end_hour=20 ) home.head()
Result output:
Residence identification
#Identify work place work = tbd.mobile_identify_work(stay, col=['user_id', 'stime', 'etime', 'LONCOL', 'LATCOL','lon','lat'], minhour=3, start_hour=8, end_hour= 20,workdaystart=0, workdayend=4) work.head()
▲ Workplace identification results
Draw activity diagram
In order to deepen our understanding of the specific activities of mobile phone users, we can use the tbd.mobile_plot_activity method provided by TransBigData to draw and observe the users’ daily activities. The specific code is as follows:
#Draw a user’s activity diagram, different colors represent different activities uid = 'fcc3a9e9df361667e00ee5c16cb08922' tbd.mobile_plot_activity(stay[stay['user_id']==uid],figsize = (20, 5))
Output result:
▲ Activity diagram of a single user
The picture above shows the activities of a mobile phone user every day during the observation period. The abscissa is the date and the ordinate is the time. Activities at the same location are displayed in the same color. From the activity diagram, we can clearly see the start and end time of each activity of this user.
Interested friends will receive a complete set of Python learning materials, including interview questions, resume information, etc. See below for details.
1. Python learning routes in all directions
The technical points in all directions of Python have been compiled to form a summary of knowledge points in various fields. Its usefulness is that you can find corresponding learning resources according to the following knowledge points to ensure that you learn more comprehensively.
2. Python essential development tools
The tools have been organized for you, and you can get started directly after installation! 
3. Latest Python study notes
When I learn a certain basic and have my own understanding ability, I will read some books or handwritten notes compiled by my seniors. These notes record their understanding of some technical points in detail. These understandings are relatively unique and can be learned. to a different way of thinking.
4. Python video collection
Watch a comprehensive zero-based learning video. Watching videos is the fastest and most effective way to learn. It is easy to get started by following the teacher’s ideas in the video, from basic to in-depth.
5. Practical cases
What you learn on paper is ultimately shallow. You must learn to type along with the video and practice it in order to apply what you have learned into practice. At this time, you can learn from some practical cases.
6. Interview Guide
Resume template
If there is any infringement, please contact us for deletion.