| Category | Technology/Tool Name | Description | Best Use Cases |
| Machine Learning Frameworks | PyTorch | Open-source deep learning framework for training and deploying LLMs. | Training and fine-tuning models like GPT, LLaMA, and Mistral. |
| TensorFlow | Google’s deep learning library with TPU support. | Training large-scale models using TPUs. |
| JAX | High-performance ML framework optimized for large-scale models. | High-speed training with automatic differentiation. |
| Distributed Training | DeepSpeed (Microsoft) | Optimized distributed training with ZeRO memory optimization. | Efficient LLM training with reduced memory footprint. |
| Megatron-LM (NVIDIA) | Model parallelism framework for transformer models. | Scaling training across thousands of GPUs. |
| FSDP (Fully Sharded Data Parallel) | Shards model parameters across GPUs to save memory. | Training extremely large models without running out of memory. |
| Data Processing | Apache Spark | Distributed data processing framework. | Large-scale dataset preprocessing before training. |
| Dask | Parallel computing framework for handling large data. | Data pipeline optimization for machine learning workloads. |
| Hugging Face Datasets | Prebuilt datasets and tools for NLP data processing. | Quickly sourcing and managing datasets for training LLMs. |
| Model Optimization | LoRA (Low-Rank Adaptation) | Fine-tuning technique that reduces computational overhead. | Efficient fine-tuning of LLMs on limited resources. |
| Quantization (GPTQ, AWQ, BitsandBytes) | Reducing model size while maintaining accuracy. | Speeding up inference and reducing memory footprint. |
| Pruning & Sparsity (Hugging Face PruneBERT) | Removing redundant model parameters to improve efficiency. | Deploying smaller, faster LLMs. |
| Inference & Deployment | Triton Inference Server (NVIDIA) | Optimized inference serving for LLMs. | High-performance, scalable LLM deployment. |
| vLLM (Fast Inference) | Memory-efficient inference server for transformer models. | Low-latency, high-throughput text generation. |
| TensorRT | NVIDIA’s inference optimization library. | Reducing inference time for real-time applications. |
| ONNX Runtime | Cross-platform AI model execution. | Running LLM models on multiple hardware backends. |
| Orchestration & Scaling | Kubernetes | Container orchestration platform for AI workloads. | Deploying LLMs in scalable cloud environments. |
| Ray | Distributed computing framework for ML scaling. | Running parallel model training and hyperparameter tuning. |
| Monitoring & Logging | MLflow | Experiment tracking and model versioning. | Keeping track of different LLM training runs and results. |
| Prometheus + Grafana | System monitoring and visualization. | Tracking GPU utilization, latency, and memory usage. |
