For information on how to use Puppeteer to build automated testing projects, you can refer to the Getting Started Guide: Using Puppeteer for Front-end Automated Testing. This article is intended to share a simplifying technique when writing test code. Applicable scenarios It is suitable for B-side system pages with many form operations and relatively uniform […]
Tag: puppeteer
Docker deployment guide for running Puppeteer on Linux server
Continuing from the previous article, Puppeteer headless browser: Open the door to automation and master the infinite possibilities of the browser world. The article introduces to you what Puppeter headless browser is, and also mentions that I had a tough time in cloud server Docker deployment. This article records and shares how I solved it. […]
Build an automatic tag generator for blog content using Puppeteer
Introduction Tags are a type of metadata used to describe and classify blog content. It can help readers quickly find topics of interest and can also improve the search engine optimization (SEO) of the blog. However, manually adding appropriate tags to each blog post is a time-consuming and labor-intensive task, and is sometimes easy to […]
How to use Puppeteer for news website data scraping and aggregation
Introduction Puppeteer is a Node.js-based library that provides a high-level API to control the Chrome or Chromium browser. Through Puppeteer, we can realize various automated tasks, such as web page screenshots, PDF generation, form filling, network monitoring, etc. This article will introduce how to use Puppeteer to capture and aggregate news website data, taking NetEase […]
How to use Puppeteer for news website data scraping and aggregation
Introduction Puppeteer is a Node.js-based library that provides a high-level API to control the Chrome or Chromium browser. Through Puppeteer, we can realize various automated tasks, such as web page screenshots, PDF generation, form filling, network monitoring, etc. This article will introduce how to use Puppeteer to capture and aggregate news website data, taking NetEase […]
Use Puppeteer to improve the accuracy and effectiveness of social media data analysis
Introduction Social media are one of the most popular platforms on the Internet, and they contain a large amount of user-generated content such as text, images, videos, comments, etc. These contents are of great value for analyzing user behavior, public opinion, market trends, etc. But how do you get this data from social media? One […]
Extracting structured data from web pages: Advanced tips with Puppeteer and Cheerio
Introduction Web scraping is a technique for extracting useful information from web pages, which can be used for various purposes such as data analysis, competitive intelligence, content aggregation, etc. However, crawling web data is not an easy task because the structure and content of web pages may change at any time, and some websites will […]
How to Implement Dynamic Web Scraping on a Node JS Server Using Puppeteer
Introduction Dynamic web crawling refers to obtaining dynamically generated data on web pages by simulating browser behavior, such as content rendered by JavaScript and data requested by Ajax. The difficulty of dynamic web crawling lies in how to deal with asynchronous events on web pages, such as clicking, scrolling, waiting, etc. Puppeteer is a Node […]
Open source online picture designer, supports PSD analysis, AI cutout, etc., and generates pictures based on Puppeteer
Github open source address: palxiao/poster-design Project Overview git clone https://github.com/palxiao/poster-design.git cd poster-design npm run prepared # Quick installation dependency instructions npm run serve # run locally The front-end interface and the image generation service (3000 and 7001 ports) will run at the same time, and a Chrome browser instance will be started locally when the […]
How to migrate business code from Puppeteer to Padchat
Wechaty has been greatly upgraded recently, and since version 0.14, it has gradually supported various non-web version solutions. My business logic code is heavily dependent on wechaty, so since wechaty released version 0.15, I have gradually migrated our business logic code from the web version to the non-web version. This blog mainly introduces how I […]