|
|
LLM Finetune Research
Qualifications(要求):
- Solid experience with PyTorch.
- Familiar with LM (BERT/T5/GPT/LLAMA, etc.) and model quantization.
- Knowledge of LoRa/QLoRa/P-tuning should be preferred.
- Experience with Triton should be preferred.
- Self-motivation, independent quick learning and logical thinking.
- Internship of at least 6 months.
Job description(工作內(nèi)容):
- finetune LLM for specific conversation scenario and modify model structure based on scenario requirements
- parameter tuning during training
- enable Parameter-Efficient Fine-Tuning for Chinese LLM
- optimize LLM training performance
Model Parallelism Performance Analyze and Optimize
Qualifications(要求):
- Strong implementation skills in at least one of C++, C, Python.
- Solid experience with PyTorch DDP.
- Familiar with cuda/dpcpp development and relative profiling tools (Nsight Systems/Vtune).
- Experience with Megatron-LM/DeepSpeed/FSDP should be preferred.
- Good at team working.
- Internship of at least 6 months.
Job description(工作內(nèi)容):
- analyze performance based on different model parallelism implementations.
- advise for model/data parallelism configuration at target environment (GPU and CPU).
- develop projection model to estimate training performance under target environment.
- implement/optimize kernel based on cuda/Triton/oneDNN.
每周五天,實(shí)習(xí)期六個(gè)月以上
地址:融科資訊中心(北四環(huán) 近中科院)
簡(jiǎn)歷投遞郵箱:qian.gong@intel.com;請(qǐng)以“姓名+學(xué)校+應(yīng)聘崗位”為郵件主題
|
|