Tag: inference

Original: flagship game graphics card (980Ti, 1080Ti, TiTan V, 2080Ti, 3090, 4090), AI drawing stable diffusion) and large model (chatglm2-6b) inference performance are rated highly

Foreword: Since the release of ChatGPT and stable diffusion last October triggered a wave of generative AI, but many people, especially some leaders, have been talking about artificial intelligence at conferences all day It brings various risks and shortcomings. In fact, I have almost never used ChatGPT and stable diffusion, and I only have a […]

deeplearning4j training inference case 2023 – handwritten digit recognition

Article directory 1.minist data set 2. Dependency package 3. Handwritten digit training and reasoning 4. Extended reading deeplearning4j comes with learning case project deeplearning4j-examples 1.minist data set Download link 6W training set, 1W test set 2. Dependency package Mainly some packages of deeplearning4j and javacv. The jar package generated in the case is 1.3G. The […]

High-performance inference using TensorRT-LLM

After the popularity of LLM, NVIDIA also released its related inference acceleration engine TensorRT-LLM. TensorRT is a high-performance deep learning inference SDK from nvidia. This SDK contains a deep learning inference optimizer and runtime environment that provides low latency and high throughput for deep learning inference applications. TensorRT-LLM is an accelerated reasoning library further optimized […]

PyTorch JIT and TorchScript, one API improves inference performance by 50%

PyTorch supports two modes: eager mode and script mode. The eager mode is mainly used for model writing, training and debugging, and the script mode is mainly for deployment, which includes PytorchJIT and TorchScript (a serialization code format that executes efficiently in PyTorch). The script mode uses torch.jit.trace and torch.jit.script to create an intermediate representation […]

OpenMMlab exports swin-transformer model and uses onnxruntime and tensorrt for inference

Export onnx file It is very easy to export the onnx file of swin-transformer through mmpretrain. Note that you need to set opset_version=12. This is a pitfall. The opset_version=11 set at the beginning will cause problems when converting trtengine later. import torch from mmpretrain import get_model, inference_model model = get_model(‘swin-tiny_16xb64_in1k’, pretrained=’swin_tiny_224_b16x64_300e_imagenet_20210616_090925-66df6be6.pth’, device=’cpu’) input = torch.zeros(1, […]

18.Template parameter type inference and reference in C++

Article directory Template parameter type inference and reference in C++ Inferring type from lvalue reference function parameter Inferring type from rvalue reference function parameter std::move–an instance of rvalue reference template function Parameter forwarding Welcome to personal network logZhixing Space Template parameter type inference and reference in C++ Inferring type from lvalue reference function parameter The […]

[SegmentAnything Practical Combat – Inference 3] Interactively select prompt boxes and prompt points to complete SAM inference on your own data set

prompt boxes + points joint reasoning write in front Effect + code 1. Select target box (1) + points for prediction 1.1 Effect All code that can be run Complete code Summarize Generally speaking, if you want to fully understand the SAM model code, the most important thing is to take a good look at […]

[SegmentAnything Practical Combat – Inference 1] Interactively select prompt points to complete SAM inference on your own data set

prompt points reasoning write in front Effect + code 1. Prediction using multiple prompt points on a single picture 1.1 Select points 1.2 When multimask_output = False, only one mask is output. 1.3 When multimask_output = True, three masks are output 2. Prediction using multiple prompt points on multiple pictures 2.1 Select points (take selecting […]