Introduction to scaling Large Model training and inference using DeepSpeed
DeepSpeed library optimizes training and inference for foundational models. Learn more about its features.
Machine Learning
Understand CLIP (Contrastive Language-Image Pre-Training) — Visual Models from NLP
CLIP introduces a model that enables zero shot learning for a new dataset (in addition to a new example) by using NLP to supervise pre-training. i.e., To identify an object, you can provide the name or description of a new object that the model has not seen before. Traditionally a computer vision model was trained … Read more
Batch Normalisation
Before we describe what Batch Normalisation is, here are a few introductory terms Internal covariate shift Stochastic Gradient Descent uses a minibatch of input to train the parameters of a layer. The input to a layer is the output from the previous layer. A change in the parameters of the previous layer causes a change … Read more
Distributed Machine Learning – Part 2 Architecture
Why Distributed Machine Learning? In the previous article we looked at how GPGPU, ASICS, AWS’s Inferentia, the new NVidia A100 chip and other advances in hardware have tremendously improved the performance of Machine Learning training and inference. However the increase in the volume of data and the increasing complexity of the machine learning models require … Read more
Distributed Machine Learning – Part 1 Hardware
Machine learning algorithms vary in size from a few parameters to a few billion parameters (e.g. GPT-3). The training data ranges from a few hundred training row to millions of rows. Training a model on a single CPU is not always efficient and so people started using GPU. GPU vs CPU vs GPGPU Wait a … Read more
Create/drop Athena table using python in Sagemaker
Amazon Athena is a serverless query engine that allows you to query data in Amazon S3 using standard SQL. You can also use the query result to create another table. This table can then be used as a source of further analytics or reporting. For example : You can use Athena to perform feature engineering … Read more
CloudFormation for SageMaker instance
Amazon SageMaker helps data scientists and Machine Learning developers build, train and deploy machine learning models. It includes Jupyter notebook to build and train model as well SageMaker API to train and deploy model with a few lines of code. Amazon CloudFormation helps in provisioning AWS resources using code. It automates provisioning and configuring resources … Read more
Statistics – Examples
T-Test:Problem : A swimming instructor wants to prove that the swimming speed of an athlete increases if the athelete performs some specific exercises before the swim. He undertakes an experiment with 16 participants and randomly assignes 8 participants to each team. For team A he recommends some common exercises and for team B he recommends … Read more
Use of Statistics – Practical Considerations
It is often confusing to decide on which statistic to use at what point. Also researchers need to be careful that the statistics they present does truly apply in the context of the problem. Statictics can be misleading and probably incorrect if used outside the boundaries set by its assumptions. In this post we analyse … Read more
Non Parametric Statistics
The earlier posts on inferential statistics show methods that work on data whose population parameters and distribution are known. However, there are cases where the population parameters are not known. In such cases, no assumption can be made about the population statistics and hence parametric methods cannot be used. In such cases there are certain … Read more