Python+Pickle/Parquet/HDF5…Comparison of quantization factor calculation performance under different file format storage modes

In quantitative trading, high-frequency factor calculation based on financial market L1/L2 quotations and transaction high-frequency data is a common investment research requirement. As the amount of financial market data continues to increase, traditional relational databases have been unable to meet the storage and query needs of large-scale data. In order to cope with this challenge, […]

Comparison of Python + HDF5 factor calculation and DolphinDB integrated factor calculation scheme

In quantitative trading, it is a very common investment research requirement to perform high-frequency factor calculation based on L1/L2 quotations and high-frequency trading data in the financial market. At present, the L2 historical data of the domestic market for ten years is about 20 ~ 50T, and the amount of new data added every day […]

NCL reads hdf5 files–taking AMSR2 sea ice density data as an example

Since most calculations are performed on the server. Therefore, it is considered to improve the level of NCL and Shell scripts. The previous blog post Python reads HDF5 and introduces HDF5 files and its reading and writing. However, due to the old system version of the server itself and the reason of the intranet, it […]