COPS IIT(BHU)

How to use multiple GitHub accounts on single machine

Wed, 27 May 2020 00:00:00 +0000

Most of us have multiple (mostly two) GitHub accounts, personal and work account. You need to have the ability to push and pull to multiple accounts. This post is about how to setup and use them on a single machine using HTTPS or SSH.

Note: The instructions below have all been executed on macOS and should work fine on all Unix based operating systems.

PS: This blog is originally published on https://jogendra.github.io/

Using HTTPS

Step 1: Change remote URL

git remote set-url <remote-name> https://<username>@github.com/<username>/<repo-name>.git

If you are setting-up new remote:

git remote add <remote-name> https://<username>@github.com/<username>/<repo-name>.git

Step 2: Update git config

By default, git use a system-wide configuration file (global config file) or the one stored at top of your home directory to pick your username and email. To ensure that the commits appear as performed by correct username, you have to setup the user.name and user.email for project, too:

git config user.name <username>
git cofig user.email <your-email@example.com>

Every git repository has a hidden .git folder (dotfile) which stores all of git related information. You can always check the username and email for a particular project.

open .git/config // check [user] section for email and username

You are good to go now!

git push <remote> <branch>

Entering this, terminal will ask your GitHub account password (once). If it is asking for password each time you push commit, you can cache them using git credential store:

git config credential.helper store

Using this helper will store your passwords unencrypted on disk, protected only by filesystem permissions. This command stores credentials indefinitely on disk for use by future Git programs.

Using SSH

Step 1: Generate new SSH Keys

Before generating new SSH keys, check for existing SSH keys. To list existing ssh keys, go to the terminal, run command ls -al ~/.ssh, files with extension .pub are your SSH keys. You need to have ssh keys for each account. If you are planning to use two accounts personal and work, there should be two SSH keys, if not, you have to generate them.

If you see No such file or directory after running ls -al ~/.ssh command, go ahead and create ~/.ssh directory with command mkdir -p ~/.ssh.

Generate SSH key for Work account

To generate a new SSH key, go to terminal and run the command:

ssh-keygen -t rsa -C <work-email>

To get the email associated with your work account:

Go to GitHub
Log in to your work account
Navigate to Settings
Copy the associated email

On running the command, you will see:

Generating public/private rsa key pair.
Enter file in which to save the key (/Users/<current user>/.ssh/id_rsa):

Enter file name /Users/<current-user>/.ssh/id_rsa_work. Default will be id_rsa. But to differentiate between work and personal, use id_rsa_work_ and id_rsa_personal. After entering the key file name, you will be asked for entering passphrase.

Enter passphrase (empty for no passphrase):
Enter same passphrase again:

A passphrase is similar to a password. The purpose of the passphrase is usually to encrypt the private key. This makes the key file by itself useless to an attacker.

Generate SSH key for Personal account

Follow the same steps as generating an SSH key for the work account. Save the key file as id_rsa_personal.

After generating SSH keys for both work and personal account, if you enter ls -al ~/.ssh, you will see list of your keys as below:

-rw-------   1 <current-user>  staff  2655 Apr  5 02:18 id_rsa_work
-rw-r--r--   1 <current-user>  staff   579 Apr  5 02:18 id_rsa_work.pub
-rw-------   1 <current-user>  staff  2655 Apr  5 02:11 id_rsa_personal
-rw-r--r--   1 <current-user>  staff   571 Apr  5 02:11 id_rsa_personal.pub

id_rsa_work.pub and id_rsa_personal.pub are your public SSH keys. id_rsa_work and id_rsa_personal (keys without .pub extension) are private keys. SSH keys always come in twos. The private key is stored on the client (your machine). The public key is stored on the remote machine. When you makes an attempt to connect to a remote machine (GitHub here but true in general) via SSH, the SSH protocol will check your computer for the private key that matches the public key stored on the remote machine. If they matches, the connection is successful.

Step 2: Add generated SSH Keys to GitHub accounts

For Work account

Copy your work account SSh key to machine clipboard with command:

pbcopy < ~/.ssh/id_rsa_work.pub

To associate SSH key with GitHub account:

Go to GitHub
Login to your work account
Navigate to Settings
Go to SSH and GPG Keys section from the sidebar
Click on New SSH Key button
Paste key in Key text field.

You will most likely receive a mail from GitHub that a new public key is added to your account.

For Personal account

Use key id_rsa_personal and follow the same step as work account for personal account.

Step 3: Update git global configuration

Ignore this step if you are going for Step 4

Git global configuration file should be aware of all of your GitHub accounts. Run this command to edit the global config file:

git config --global --edit

Config file will open in vim, enter i for insert mode and enter:

[work]
  name = <work github username>
  email = <work email>
[personal]
  name = <personal github username>
  email = <personal email>

Press esc and :wq to exit the insert mode in vim.

Now, whenever you will be pulling or pushing, you will be asked which account should be used for this repositopry (once only) and the local config file (repository level) will remember the account.

Important: Make sure you’re using SSH instead of HTTPS as remote URL:

git remote set-url <remote-name> git@github.com:<username>/<repo-name>.git

If you are setting-up new remote:

git remote add <remote-name> git@github.com:<username>/<repo-name>.git

Now, you are all set to use multiple GitHub accounts on your machine!

Further, feel free to use SSH configuration rules explanined below if you find them more handy.

Step 4: Add SSH configuration rules [OPTIONAL]

Register new SSH keys

Before configuring rules, register the new SSH Keys with the ssh-agent. Open the terminal and run:

ssh-add ~/.ssh/id_rsa_work
ssh-add ~/.ssh/id_rsa_personal

If you entered passphrase during generating key process, you will be asked to enter the same here, not otherwise.

Enter passphrase for /Users/<current-user>/.ssh/id_rsa_work:

(same for personal account key)

On success you will see:

Identity added: /Users/<current-user>/.ssh/id_rsa_work (<work-email>)

(same for personal account key)

To make sure if ssh-agent is running, run command eval "$(ssh-agent -s)".

Create SSH Config File and Add Rules

Create new SSH config file by runnning

touch ~/.ssh/config

We need to add SSH config rules for different hosts to specify which identity file to use for which domain. This is way to tell SSH when to use work account and when to use personal account while performing git/github related stuff.

Open newly created config file with command vim ~/.ssh/config. You can choose other editor to edit file if you are not comfortable with vim.

Add rules to config file as below:

# Rule for Work account
Host github.com-work
   HostName github.com
   User git
   IdentityFile ~/.ssh/id_rsa_work

# Rule for Personal account
Host github.com-personal
   HostName github.com
   User git
   IdentityFile ~/.ssh/id_rsa_personal

Test the rules

To check if everything is working fine, create a new test repository on your personal GitHub account and run commands below in terminal:

mkdir TestRepo
cd TestRepo
git init
touch testfile.txt
echo "This is test line" >> testfile.txt
git add testfile.txt
git commit -m "This is test commit"
git remote add origin git@github-personal:<personal-username>/TestRepo.git
git push origin master

Things to be taken care

If you have the repository already cloned update remote URLs like below:

git remote set-url <remote-name> git@github.com-personal:<username>/<repo-name>.git

If you are creating a new repository set remote like below:

git remote add <remote-name> git@github.com-personal:<username>/<repo-name>.git

If you are cloning a repository

git clone git@github.com-personal:<username>/<repo-name>.git

(same for work repositories, use github.com-work)

All set, rest will be taken care by SSH configuration!

Don’t forget to share ^_^ You can find me on Twitter.

AR Portals using 8th wall

Tue, 12 Nov 2019 00:00:00 +0000

AR Portals using 8th wall

8th wall web:

Web AR for mobile devices!

Built entirely using standards-compliant JavaScript and WebGL, 8th Wall Web is a complete implementation of 8th Wall’s Simultaneous Localization and Mapping (SLAM) engine, hyper-optimized for real-time AR on mobile browsers. Features include 6-Degrees of Freedom Tracking, Surface Estimation, Lighting, World Points and Hit Tests.

Web AR Examples

A-Frame Examples(Recommended to start)
Babylon.js Examples
three.js Examples
Camera Pipeline Examples
Amazon Sumerian Setup

Prerequisites

install npm

Getting Started

Head over to 8th wall website and sign up if you haven’t.
Go to device authorization and authorize your browser and phone (through QR code scan). You will see developer mode “ON”.
Go to settings and grab app key. We will use that later…
Well done! :thumbsup:

Github :octocat:

Go to Github 8th wall/Web
Git Clone Web repository so you have it in your local.

so far so good…

Follow these steps

Go to examples/aframe/portal/index.html
Replace app key on line 15.
Then cd into the serve folder.
npm install
cd .. to reach the root of the repository.
Let’s start the server now. Type ./serve/bin/serve -d ./examples/aframe/portal/ -p 7777 -i en0

Steps to guide

Custom Modifications

“City.mp4” 360 video link: https://cdn.8thwall.com/web/assets/video/city.mp4

Script:

AFRAME.registerComponent('play-on-window-click', {
  init: function() {
    this.onClick = this.onClick.bind(this);
  },
  play: function() {
    window.addEventListener('click', this.onClick);
  },
  pause: function() {
    window.removeEventListener('click', this.onClick);
  },
  onClick: function(evt) {
    var video = this.el.components.material.material.map.image;
    if (!video) { return; }
    video.play();
  }
});
// Add this to line 24 before the ending of script tag.  

Serve Command:

Make sure you are in the “web-master” directory before executing the following command. (run “pwd” on Mac or “cd” on Windows to print out the current working directory)

Mac:

./serve/bin/serve -d examples/aframe/portal

Windows:

serve\bin\serve -d examples\aframe\portal

Note: If you are trying to run the serve script from another directory, you’ll likely have to adjust the paths to the serve script and/or project directory. For example, if you are in the “web-master/serve/bin” directory, you’d run:

Mac:

./serve -d ../../examples/aframe/portal

Windows:

serve.bat -d ....\examples\aframe\portal

Asset Declaration:

<video id="video" style="display:none"
    autoplay loop crossorigin="anonymous" playsinline
    webkit-playsinline>
  <!-- MP4 video source. -->
  <source type="video/mp4" src="NAME.mp4" />
</video>
// Add this inside <a-assets> on line 94  

// Edit NAME.mp4 with city.mp4

Call Asset:

 <a-videosphere rotation="0 0 0" src="#video" play-on-window-click>
 </a-videosphere>
 // Add this code to line 173 and remove the unicorn.gltf and the platform over which it's placed.   

Thank YOU

CSOC ML Week5

Wed, 26 Jun 2019 00:00:00 +0000

Introduction

Hello and welcome to week 5 of CSOC ML although it is not exactly that but let us go on with it. Hope that you have gone through most of the resources given to you till now and have found the resources helpful. This week we will start with some advanced topics like Neural Networks and Convolutional Neural Networks. The main aim from this week onwards will be to give you a general idea about the modern state of Machine Learning and provide you the resources which will act as starting points to explore and get an insight of the field. The materials provided may be too much to cover in just weeks(they will be!!) but as mentioned above these will act as a guide to you in the future so don’t force yourself with completing everything within 2 weeks!!

Courses

Before we start I will be providing a list of awesome courses that you can do in these topic. These courses will give you a lot of knowledge about the field.

The first two courses give you a strong idea and intuition about Neural Networks and Convolutional Neural Networks. The third is a more comprehensive and exhaustive course on both the subjects by Stanford University. Once you complete the course on Neural Networks(1st course), the third course is highly recommended. Take your time to go through these courses.

Neural Networks

Now coming to the more meatier part, what are neural networks??? Neural networks are a set of algorithms, modeled loosely after the human brain, that is designed to recognize patterns. This video will make things much clearer and give you an intuitive idea about neural networks. Now, the question arises how do the neural networks learn to identify or recognize patterns? These three videos will give you a clear idea about this:

Also for more insights go through the first two chapters of this book. The two chapters explain things more mathematically with the appropriate code. So people who want strong mathematical justification(which should be all of you guys) this is a very good resource.

Convolutional Neural Networks

If you have grasped the concept of neural networks clearly then understanding Convolutional Neural Networks(CNN in short) would not be hard. CNN’s are widely used with the data involved is images, although they have several other applications. The following blog will make things clearer:

These blogs and videos will help you get good intuition about Convolutional Neural Network. We will be providing some more resources on this the next week. Go through the Stanford course and Andrew Ng’s course mentioned above for the more in-depth understanding of the concept and to get insight into more useful applications.

Programming libraries

Now one question you may all have is “Do I have to code the Neural Networks and CNN’s from scratch along with backpropagation etc..”. The obvious answer is NO!!. There are a lot of programming frameworks that do it for you and if you are considering going into this field knowledge about any one of these is a must. We will discuss two famous and widely used frameworks, Tensorflow and Pytorch.

Tensorflow

Many of you must have heard about this. TensorFlow(https://www.tensorflow.org/) is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets you easily build and deploy ML powered applications. Tensorflow has changed a lot over time and its present integration with another deep learning framework Keras has made it much easier to work on it. There are several tutorials on tensorflow available online. So just Google. However to get started, go through this basic tutorial. More can be found on their official page.

Pytorch

Pytorch(https://pytorch.org/) is another famous library for Deep learning. It is an open source deep learning platform that provides a seamless path from research prototyping to production deployment. The best thing about Pytorch is it is pythonic and hence very easy to learn. The best way to learn pytorch is through the official tutorials. These are extremely good tutorials. This blog also provides some great insights. Also, you can get several more resources on this just by Googling

Colab

Bigger neural networks mean more and more computation and more and more time. Hence GPU’s (Graphics processing unit) are a must if you want to train your networks fast. This blog will explain the benefit of using a GPU and why they are essential for Deep learning. Now all of your systems may not have an Nvidia Graphics card or may not have the requirements to smoothly run ML algorithms. DO NOT WORRY!!. Colab provides you free GPU’s!! Seriously!!. For anyone who doesn’t already know, Google has done the coolest thing ever by providing a free cloud service based on Jupyter Notebooks that support free GPU. You can use colab to run your ML algorithms and networks. All you need is a google account which is also free!!.

This article will help you understand Colab and its functionalities. Make yourselves familiar with this platform as all the assignments in the coming weeks will be on Colab.

Well, a lot of material has been provided to you. Do not rush through it go through them slowly and if you get stuck anywhere ask your doubts in any of the various groups and get them cleared. These materials cover most of the import concepts in the field. So start with the first course on Neural Networks by Andrew Ng and then go through all the blogs and videos given in The Neural Networks and Convolutional Neural Networks section. Start with Tensorflow and make yourself familiar with Colab. This will be sufficient for the next assignment. Meanwhile, continue to go through the above-mentioned courses. Also, all those who have submitted the assignment will get reviews through the email they have provided. So, ALL THE BEST!!

CSOC ML Week4

Sun, 09 Jun 2019 00:00:00 +0000

Hello and welcome to the week 4 of CSOC ML. Hope that you have gone through most of the resources given to you till now and have found the resources helpful. The resources cover the basics of Machine Learning and Data Science. We also hope you have completed the assignment 2 sincerely. Also the third assignment has been released. Today there wont be any more resources but we will discuss the assignment that you will be working on this week and the final task for the Data Science part of this programme. Continue going through whatever that has been provided to you untill now and complete it sincerely.

Assignment

There are no tasks in this assignment like the previous ones. We have provided you a dataset on which you will be working on. You have to analyse the data set and build models to get good prediction the data set. If you have gone through all the resources that has been provided to you untill now especially the blogs, you will have a clear idea as to how to do this. This task will tell us how much sincerely you have gone through the resources. More instructions are given in the jupyter nootebook. Remember do not copy from publicaly available solutions as this is to test your knowledge and for you to evaluate yourselves. You will have to explain the results you are getting and why you have done so. So blindly copying isnt going to help. There will be a form released soon where you will have to fill all these details.

Final Task

As promised in the previous blog the final task will be related to Kaggle. The Titanic: Machine Learning from Disaster is a very famous and a begineer competition on Kaggle. In this competition you are required to apply the tools of machine learning to predict which passengers survived the tragedy. You are required to solve the challenge and make a submission on Kaggle. Take a screenshot of your submission i.e your score ,upload the code in your drive and fill the google form. If you have used Kaggle to write the code then provide the link of your notebook. You have time till June 30 midnight to complete this task. Again dont copy from others’ solution. Refer them to understand the various approaches that can be used and try to come up with your own solution.

This week marks the end of Data Science part of the programme. From next week onwards we will start with some advanced topics and dive into the amazing field of Computer Vision. Untill then ALL THE BEST!!!

CSOC Dev - Week 3

Sat, 08 Jun 2019 00:00:00 +0000

Hello and welcome to the week 3 of CSOC Dev. This week will cover essential web backend development using the popular web framework - Django!

What is Django?

A Python-based web framework for ‘perfectionists with deadlines’, it is a battle-tested framework used to make websites from personal website to web apps for big companies like Instagram. One of its core principles is DRY and promotes building a web app using MVC architecture. This article is helpful to understand its project structure.

Understanding HTTP and some of its methods are a must - HTTP methods

Any tutorial?

There’s lots of community around Django, and one of the most famous ones is Django Girls. Here’s a tutorial from them: Django Girls Tutorial

For authentication in Django, study from this article. Try out logic of custom login in Django from here

CSOC Dev - Week 2

Fri, 31 May 2019 00:00:00 +0000

Hello and welcome to the week 2 of CSOC Dev. I hope all of you liked messing around with branches and stuff. Did anyone try making their Octocat yet? :D. This week will cover essential web frontend development using the fundamentals of fundamentals - HTML!

What is HTML?

No, it’s not a programming language first of all. Don’t even plan to write it in your CV as such :D. It’s a markup language to structure your document and it’s not constrained to be linear. That’s what hypertext means in the full form of HTML - Hypertext Markup Language.

Can you be more practical?

Okay, let’s start with useful stuff! To make any good-looking site of yours, which you want to admire and fawn over, you need three things - HTML, CSS and JS. Any front-end developer requires these three masterpieces.

Now, when you get started to learn from the internet, there are millions of resources to learn. Let’s filter out some to learn only relevant stuff, not how to make your webpage run on old Netscape Navigator :D. First, learn anything which concerns with HTML5, CSS3 and JS (ES 5+). Those aren’t the latest; especially the JS has undergone lots of changes in ES6 specs.

Simplest site to learn basics of web dev is no doubt w3schools. So, get started on HTML and CSS from there. For CSS, don’t get lost there. Just learn syntax, how to use it and then keep the following references handy:

HTML references
CSS references
When you don’t understand something in CSS, remember to search here: CSS tricks
God of answers :D

For JS, it’s very, very vast. Start with this crash course first: https://developer.mozilla.org/en-US/docs/Learn/Getting_started_with_the_web/JavaScript_basics

And to practice using with your HTML page, read DOM Manipulation: https://www.sitepoint.com/dom-manipulation-vanilla-javascript-no-jquery/

CSOC ML Week3

Thu, 30 May 2019 00:00:00 +0000

Hello and welcome to the week 3 of CSOC ML. Hope all of you are going through the resources given last week and have submitted the first assignment. The next two weeks we will be learning about Data Science. The topic is too huge to cover in two weeks but we will try to give you a very good idea about the field. Also the second assignment on matplotlib and pandas has been released and the solution to first assignment will soon be released. You have time till next Wesdnesday midnight to submit the assignment. Do not forget to fill the google form after completing the assignment. Now coming to the resources of this week.

What is Data Science?

Data Science is a huge field in itself. Its not just making complicated models and vizualization but its using data to make as much impact as possible. Here is an amazing video in which a data scientist talks about what actually is data science. This blog will also give you a good idea about the various aspects of Data Science.

Kaggle

If you are intrested in data science then kaggle is the place for you. Kaggle is platform to compete with others in competitions which are based on machine learning tasks. You may know about Codechef, Hackerrank etc., so kaggle is also like them, but the key difference is the competition are only related to machine learning, data science, Deep learning or AI related. There are enormous amounts of resources, tutorials, datasets and code available on Kaggle. Its an amazing place to gain knowledge and start with compititive ML. With time kaggle will teach you a lot about Data Science and Machine Learning in general. Sign up on Kaggle and start exploring it. The interface is extremely simple and self explanatory. This blog will clearly explain and guide you through various aspects of kaggle. Your final assignment for the first part of CSOC ML will be related to kaggle so get comfortable with the platform.

Resources

Hope that all of you are going through the courses given to you last week. After you complete them go through these:

Week 6(complete), Week 8(Dimentionality Reduction) and Week 10(complete) of Andrew Ng’s course.
Chapter 10 to 15 - Feature Scaling, Text Learning, Feature Selection, PCA, Validation, Evaluation Metrics of Intro to Machine Learning - Udacity

These along with the previous lectures will give you a clear idea about analyzing data, creating and validating models when working with various data sets.

Here are some amazing resources which highlight various steps which must be undertaken which solving a data science problem. Go through them in order. These will be very helpful for solving the next week’s assignment.

How to start a Data Science project
Data Manipulation
Feature engineering
Data visualization:
Tutorial 1
Tutorial 2
Applying Machine Learning algorithms
Ensembling(advanced,optional)

It may be too overwhelming to complete soo many things and the assignments`and look into Dev and CP at the same time. Dont worry!!! Complete the lectures and go through the other resouces as much as possible. You dont have to complete everything in these four weeks(to write). Remember, understanding whatever you have learnt and practising it is more important than trying to hurry through all the materials. If you have put sincere efforts you wil definitely be able to do all the assignments in time. We will be back next week with more resources and assignment 3. So untill then ALL THE BEST and enjoy!!!

CSOC ML Week2

Wed, 22 May 2019 00:00:00 +0000

Hello and welcome to the week 2 of CSOC ML. Hope you have gone through and practiced all the materials given last week. This week we will learn about two more important libraries matplotlib and pandas. Along with this, starting this week, we will delve right into the core Machine Learning. Also the first assignment has been released. You have time till next Wednesday(29-05-2019 23:59) to submit the assignment. Do not forget to fill the google form after completing the assignment. Now coming to the resources of this week.

pandas

Machine Learning deals with data, loads and loads of them. Understanding, analysing and manuplating this data is essential before applying Machine Learning algorithms. Pandas is a powerful data science library with provides you powerful functions to manipulate and analyze the data. Hence it is very essential to have a strong grasp over some basic functions in this library. Here are some of the good tutorial which you can refer.

Matplotlib

When working with data, we usually want to visualize it to understand the underlying patterns and get insights about the data. Matplotlib is another powerful Data Science library. As the name suggests it is a plotting library which provides powerful functions to plot the data points. A very interesting example demonstrating the importance of plotting is the Anscombe’s quartet. Here are some excellent resources on Matplotlib.

Machine Learning

Machine Learning has become one of the hottest topics of discussion and research today. Most of the big firms and industries are trying to integrate Machine Learning in their technology. But what is this Machine Learning? Machine Learning is an idea to learn from examples and experience, without being explicitly programmed. Instead of writing code, you feed data to the generic algorithm, and it builds logic based on the data given. Machine Learning has several applications. Face detection, Email Filtering, Chatbots, Weather prediction, medical diagnosis are some of them. There are three kinds of Machine Learning Algorithms. They are:

Supervised Learning
Unsupervised Learning
Reinforcement Learning

Refer this amazing blog to get a good insights of these algorithms. In the coming two weeks, we will cover the basics of Supervised Learning. We will be mainly referring to following two courses:

If you have ever asked anyone as to how to start with Machine Learning you must have heard about this course. Going through the course will give you a firm understanding of Machine Learning concepts. Another intresting course is the ud-120, Intro to Machine Learning by Udacity. This course is a bit more hands-on than the former.
In the next two weeks, you need to cover:

Week 1 - Introduction, Linear Regression with One Variable, Linear Algebra Review,
Week 2 - Linear Regression with Multiple Variables,
Week 3 - Logistic Regression, Regularization
Week 7 - Support Vector Machines part of Machine Learning course by Andrew Ng
The first seven chapters of Intro to Machine Learning course on Udacity

Be sure to take the the quizzes in Andrew Ng’s course althogh the assignments are optional. In Udacity’s course make sure you do the mini projects at the end of chapters sincerely. Start with the order given above. Doing this will give you a good intution on the core Machine Learning algorithms.
There is a lot of material to cover So I suggest you start early. The assignment has also been kept short for the same reason. Dont feel down if you are feeling a bit rusty will the libraries. As you practise and implement them more and more your grasp on them will become stronger. Next week an assignment on Matplotlib and pandas will be released. Untill then All the best and happy learning!!

CSOC ML Week1

Wed, 15 May 2019 00:00:00 +0000

Welcome to the first week of CSOC ML. This will be a two month long course where we will be developing various aspects of machine learning from basic concepts to advanced ones. There are no major prerequisites for this course. Only basic programming skills and some high school mathematics - linear algebra (matrices), calculus and probability is required. There will be a new blog every week containing all the resources for the week. There will be some links of exciting blogs which will help you gain a lot of knowledge so do go through them. So lets get started!!!

Anaconda Installation

Anaconda is a python and R distribution. It aims to provide everything you need (python wise) for machine learning “out of the box”. Installing this will setup Python and all the major data science libraries and Jupyter Notebook. here are some links to help you install Anaconda on Linux and Windows

Anaconda Installation for linux
Anaconda Installation for windows
Note: If you are an linux user you can install the necessary libraries using pip. pip is a package manager for python packages. In this case it is a good practise to create a virtual environment before installing various libraries.

Jupyter Notebook

Throughout the course we will be using Jupyter Notebook for doing assignments so getting familiar with it is very necessary. Run the Jupyter Notebook by either starting the Anaconda Navigator or by typing jupyter notebook in the terminal. You can find a brief introduction of Jupyter Notebook in these tutorial.

Python

Python and its Data Science libraries are used predominantly in competitive Machine Learning. Python is used even in Machine Learning research, because of its simplicity and availability of a wide array of machine learning libraries in Python. So it is essential to have a good grasp over Python. If you have some basic knowledge of any programming language then learning python wont take long. The first nine chapters of the official documentation of Python is an excellent resource to start with. You can also watch the first 34 videos of Python series by Sentdex.Also you can practise by solving problems on Hackerrank. The links to all these resources are given below.

Numpy

The last task for the week would be to learn Numpy. NumPy is the fundamental package for scientific computing in Python. It is a Python library that provides a multidimensional array object, various derived objects (such as masked arrays and matrices), and an assortment of routines for fast operations on arrays, including mathematical, logical, shape manipulation, sorting, selecting, I/O, discrete Fourier transforms, basic linear algebra, basic statistical operations, random simulation and much more. Some references are:

Some amazing blogs to get an insight of machine learning

Here are links to some amazing blogs. Reading various blogs is a very good practise as it helps you gain immense amount of knowledge and helps to be up-to-date with the recent trends in technology.

What’s new in Java 8: Lambdas

Mon, 03 Dec 2018 00:00:00 +0000

What’s new in Java 8: Lambdas :octocat:

A hands-on introduction to Java 8’s most exciting new feature :sparkles:

By Yash Raj Bharti | Metallurgical engineering | 2018-12-03

Java 8 is here, and, with it, come lambdas. Although long overdue, lambdas are a remarkable new feature that could make us rethink our programming styles and strategies. In particular, they offer exciting new possibilities for functional programming.

While lambdas are the most prominent addition to Java 8, there are many other new features, such as functional interfaces, virtual methods, class and method references, new time and date API, JavaScript support, and so on. In this post, I will focus mostly on lambdas and their associated features, as understanding this feature is a must for any Java programmer on-boarding to Java 8.

All of the code examples mentioned in this post can be found in this github repo

What are lambdas? :octocat:

Lambdas are succinctly expressed single method classes that represent behaviour. They can either be assigned to a variable or passed around to other methods just like we pass data as arguments.

You’d think we’d need a new function type to represent this sort of expression. Instead, Java designers cleverly used existing interfaces with one single abstract method as the lambda’s type.

Before we go into detail, let’s look at a few examples.

Example Lambda Expressions

Here are a few examples of lambda expressions:

// Concatenating strings
(String s1, String s2) -> s1+s2;

// Squaring up two integers
(i1, i2) -> i1*i2;

// Summing up the trades quantity
(Trade t1, Trade t2) -> {
  t1.setQuantity(t1.getQuantity() + t2.getQuantity());
  return t1;
};

// Expecting no arguments and invoking another method
() -> doSomething();

Have a look at them once again until you familiarize yourself with the syntax. It may seem a bit strange at first. We will discuss the syntax in the next section.

You might wonder what the type is for these expressions. The type of any lambda is a functional interface, which we discuss below.

Lambda Syntax :octocat:

There is a special syntax to create and represent a lambda expression. Just like a normal Java method, a lambda expression has input arguments, a body, and an optional return value. This is illustrated here below:

input arguments -> body

Each lambda expression has two parts separated by an arrow token: the left hand side is our method arguments, and the right hand side is what we do with those arguments (that is, applying business logic). The body can either be an expression or block of code, returning a result or void.

So, in the first lambda expression, (String s1, String s2) → s1+s2, the left hand side of the arrow (→) token is our method argument list. The arguments to the method are supplied as two strings. Coming to the right hand side part, the logic that we are applying in the method is what we are expressing here.

So, from our example, all we need to do is—given two strings, concatenate them up! Whatever logic we put in a method, can go here in this block. In our example, it is adding the both input arguments: s1+s2. The right hand side is the body where we can write statements, expressions, code blocks, calling other methods, etc.

Lambda’s Type: the Functional Interface

I briefly mentioned earlier that the type of a lambda is a functional interface. As Java is a strongly typed language, it is usually mandatory to declare types; otherwise, the compiler laughs at us. Note, however, that we omitted types when declaring the above lambda expressions. So, what is the type of a lambda expression? Is it a string, an object, or a new functional type?

Fortunately, there is no new addition to the type system—the designers of Java did not introduce any special type to represent lambda expressions. Instead, they cleverly re-used the existing anonymous method strategy. We’re already familiar with anonymous classes, so picking up this new representation should be relatively straight forward.

A functional interface is a special interface with one and only one abstract method. It is exactly the same as our normal interface, but with two additional characteristics:

It has one and only one abstract method.
It can be decorated with an optional @FunctionalInterface annotation to be used as a lambda expression. (this is strongly suggested) Java has numerous single method interfaces. These are all retrofitted, making them functional interfaces. If we want to create our own, all we need to do is to define an interface with one abstract method and stick a @FunctionalInterface annotation on top of it!

For example, the following snippet defines an IAddable interface. This is a functional interface whose job is to simply add two identical items of type T.

@FunctionalInterface
public interface IAddable<T> {
    // To add two objects
    public T add(T t1, T t2);
}

Because this interface has one and only one abstract method and it is annotated with @FunctionalInterface, it can be used as a type for representing lambda functions.

Below are some example representations that use the above IAddable functional interface here:

// Our interface implementations using Lambda expressions
// Joining two strings—note the interface is a generic type

IAddable<String> stringAdder = (String s1, String s2) -> s1+s2;

// Squaring the number
IAddable<Integer> square = (i1, i2) -> i1*i2;

// Summing up the trades quantity
IAddable<Trade> tradeAdder = (Trade t1, Trade t2) -> {
  t1.setQuantity(t1.getQuantity() + t2.getQuantity());
  return t1;
};

Note that the IAddable is a generic type interface, therefore we’re using it to add different types, as in the above example.

To summarize, the lambda expression’s type is the functional interface that we intended to implement via the lambda expression.

Once we have the implementations ready, we can use them in our class by simply invoking the respective method. See the example below, which shows how the above implementations are being used.

// A lambda expression for adding two strings.
IAddable<String> stringAdder = (s1, s2) -> s1+s2;

// this method adds the two strings using the first lambda expression
private void addStrings(String s1, String s2) {
  log("Concatenated Result: " + stringAdder.add(s1, s2));
}

Before we move on, let’s take a moment to understand what is achieved here. The noticeable thing is that we are now treating the business logic as a function that can be tossed around for execution. We can define numerous variations of the business logic on the fly (kind of!) without having to create and instantiate classes as we used to do before.

Now that we understand a bit about lambda expressions and their types, let’s run through a complete example using lambdas. We’ll also compare and contrast with the pre-Java 8 version to understand the differences.

Lambda Example :octocat:

Our requirement is to create business logic for merging two trades issued by the same issuer. This example should solve the requirement by using both pre-Java 8 and Java 8 approaches.

Pre-Java 8 Implementation :sparkles:

Prior to Java 8, we were expected to implement the interface with concrete definitions, as shown in the test class below:

public void testPreJava8() {
  IAddable<Trade> tradeMerger = new IAddable<Trade>() {
  @Override
    public Trade add(Trade t1, Trade t2) {
      t1.setQuantity(t1.getQuantity() + t2.getQuantity());
      return t1;
    }
  };
}

Here, we created a concrete class implementing the interface and invoked the add method on the instantiated object. We used an anonymous strategy for creating the code shown above.

Although the actual logic of merging the trades is our core logic, we are forced to do some additional tasks such as implementing the interface with a class identity, overriding the abstract method, creating an instance of it and finally doing something with the instance. This “excess baggage” has always attracted critics and made developers uneasy—it’s a lot of boiler plate code and meaningless implementations and instantiations for a piece of business logic.

Once we have the instance of the class, we follow the usual process of invoking the respective methods on the instance, as shown below:

IAddable addable = ....;

Trade t1 = new Trade(1, "GOOG", 12000, "NEW");
Trade t2 = new Trade(2, "GOOG", 24000, "NEW");

// using the conventional anonymous class..
Trade mergedTrade = tradeMerger.add(t1,t2);

The business logic is intertwined with the technical garbage. The core logic is very much tied to the implementing class. For example, instead of returning a merged trade as in the above case, if I may have to compare and return the big trade, I have to sigh a bit, grab a coffee, sneeze loudly, moan, groan and roll up my sleeves and get ready to rewrite the code logic.

Note also that all our test cases will start failing too once we change the logic!

What if I have to support dozens of such requirements? Well, either I have to hack the current method to create the additional logic path using a control statement (if-else or switch) or create a new class for each piece of logic. Tightly coupling your business logic to its implementing class is asking for trouble—especially if we have fickle-minded business analysts and project managers. Surely there must be a better way to make it work.

Java 8 Implementation :octocat:

Creating multiple behaviors using anonymous classes is not impossible, but it’s not ideal. We can simplify this problem by using lambda functions to represent various behaviors. So, for example, we can write a lambda expression to sum up the trade’s quantities, another for returning a large trade, another expression for encrypting trade data, etc. All we do is write a lambda expression for each of the requirements and send it over to the class that expects the lambda.

For example, take a look at this list of lambda expressions for our requirements:

// Summing up the trades quantity
IAddable<Trade> aggregatedQty = (t1, t2) -> {
  t1.setQuantity(t1.getQuantity() + t2.getQuantity());
  return t1;
};

// Return a large trade
IAddable<Trade> largeTrade = (t1, t2) -> {
 if (t1.getQuantity() > 1000000)
   return t1;
 else
   return t2;
};

// Encrypting the trades (Lambda uses an existing method)
IAddable<Trade> encryptTrade = (t1, t2) -> encrypt(t1,t2);
As you can see, we have declared lambdas for each our respective functionalities. The method can now be modeled to expect an expression, as shown below:


//A generic method with an expected lambda
public void applyBehaviour(IAddable addable, Trade t1, Trade t2){
  addable.add(t1, t2);
}

Note that the method is generic enough that it can apply functionality to any two trades with the given behavior using the given lambda expression (the IAddable interface).

Now, the client has the control of creating the behaviors and pass it on to the remote server for application of them. This way, the client cares about what to do, while the server cares about how to do it. As long as the interface is designed to accept the lambda expression, the client can create a number of those expressions according to its requirements and invoke the method.

Before we sum up, let’s take an existing Runnable interface, which comes with lambda support, and see how it can be used.

Runnable Functional interface :octocat:

The most popular Runnable interface has one method, run, which takes no arguments and returns nothing. Perhaps the reasoning behind this is that a piece of logic is to be run in a separate thread to speed up the process.

The new definition of Runnable interface, followed by an example implementation using an anonymous class, is given below:

// The functional interface
@FunctionalInterface
public interface Runnable {
  public void run();
}

// example implementation
new Thread(new Runnable() {
  @Override
  public void run() {
    sendAnEmail();
  }
}).start();

As you can see the above way of creating and using the anonymous class is very verbose—and unsightly too. Apart from sendAnEmail() in the run method, everything else is redundant boilerplate code.

The same Runnable can now be re-written to use a lambda expression, as shown below:

// The constructor now takes in a lambda
new Thread( () -> sendAnEmail() ).start();

The lambda expression () → sendAnEmail() highlighted above is passed to the constructor of the thread. Note that this expression is effectively a piece of code (an instance of Runnable) that carries certain behavior (make sure to always send an email in a new thread).

Looking at the expression, we can deduce the type of the lambda—in this case it is Runnable, as we all know that the thread constructor accepts a Runnable. If you’ve noticed the redefined interface definition, Runnable is now a functional interface and hence tagged with the @FunctionalInterface annotation. Now the lambda expression can be assigned to a class’s variable as Runnable r = () → sendAnEmail() just like how we declare and assign variables.

This is the power of lambdas—they allow us to pass a behavior to a method, assigned to a variable (free floating) from another method, just like we do when passing arguments that carry data.

Let’s suppose we have a server side class called AsyncManager whose sole purpose is to execute requests on different threads. It has a single method, runAsync, which accepts a Runnable, as shown below:

public class AsyncManager{
  public void runAsync(Runnable r) {
    new Thread(r);
  }
}

The client now has the ability to create a plethora of lambda expressions based on his requirements. For example, see the various lambda expressions that can be passed on to the server-side class:

  manager.runAsync(() -> System.out.println("Running in Async mode"));
  manager.runAsync(() -> sendAnEmail());
  manager.runAsync(() -> {
    persistToDatabase();
    goToMoon();
    returnFromMars();
    sendAnEmail();
  });

Summary :octocat:

In this post, we learned about the biggest change Java has ever seen. Lambdas will certainly steer the direction of Java and make it more appealing to varied sections of the programmer community. :sparkles:

Creating an ML based Python API hosted on heroku

Sun, 15 Apr 2018 00:00:00 +0000

Creating an ML based Python API hosted on Heroku

Since long I’ve been thinking of creating an API which fellow developers could use over cloud. But I didn’t want it to be the traditional Hello World API or simple SQL Flask API — supporting the classical user name and email ID GET, PUT, POST, DELETE REST requests. Since AI and ML are so pervasive now, I thought of giving ML a try — and it is easy : )

Image Classification using PyTorch

Since everything’s on the cloud and the free versions I needed an AI/ML project with small, lightweight dependencies for the cloud. I forked a project named Img2vec that uses PyTorch to generate feature vectors for a dataset of images and then does a simple match(cosine similarity in sklearn) of a test image with others using pretrained models. The readme provides enough snippets to make the demo work on your system.

Training the image data set

For the sake of simplicity, I’m skipping the data set training part — there are lot of docs over the internet. For our API purpose, I’m generating image vectors and storing them as csv files. Since computations will happen over cloud, if we generate vectors on runtime, the API will always timeout. But the API will generate vector for the image data you’ll ping and read other vectors from csv files.

Setting up the Flask app

For a simple image match, I needed a POST or PUT request. There are many frameworks that support REST features, I used flask on Python 3.6. As the official site reads

Flask is a lightweight WSGI web application framework. It is designed to make getting started quick and easy

So here’s the idea:

The server receives an image as byte stream
Those bytes are encoded back to an image using Pillow in Python
A feature vector is generated for that image
That vector is matched against the training data images
Closest matching images are returned back (with a download URL and a percentage match value)

Here’s the gist of the API :

# endpoint to detect image
@app.route("/image_clustering", methods=["PUT"])
def image_clustering():

    # Image converted in Base64 encoded byte stream
    bytes = request.get_data()
    results = search(bytes)  # returns a list of image matches
    return jsonify(results)

Note: The test Image posted is not normal byte stream, its base64 encoded image byte stream — a standard way to ship binary data across networks.

Note: Get the source code here.

Setting up the Heroku server

Setting up the server is pretty straight forward, but there are a few issues I encountered:

Slug Size: Heroku provides a max slug size of 500 MB, the total data an app can hold(code, executables, and media files like images, pdfs) after compression. This is an issue since python libraries like SciPy and PyTorch are pretty hefty in size — PyTorch with CUDA 8 is ~600MB alone. But there are workarounds.
PyTorch Version: PyTorch provides a CPU only build variant, a small 45 MB library providing all the features we need for deployment.
SciPy Manual Install: Don’t know why, but an online install of SciPy on Heroku turns out to be buggy(more of it here. Instead I downloaded the SciPy whl(or the source code) as a file and manually installed it on the server.
JPEG vs PNG: In the demo code, I’ve used JPEG extension files(with base64 encoded data beginning from /9j/9…). So if you ping the server with images in other formats like PNG(base64 encoded data starts with iVi…) you will get an error.

Demo Time

You can take a demo in many ways

You can install an app(get it here) on your android device. Click an image it’ll take some time and will display the closest matching images.
You can use this python script. Just enter the relative path of the image on your desktop and it will display the results
You can make a PUT request online(I recommend Hurl.it for starters) to this URL with base64 encoded image data as body.

For the sample cat image shown, the results are

[
  [
    0.8155140106062471, 
    "https://www.googleapis.com/download/storage/v1/b/python-clustering-api.appspot.com/o/images%2FFace%2F124.jpg?generation=1522585329188523&alt=media"
  ], 
  [
    0.8145577585207011, 
    "https://www.googleapis.com/download/storage/v1/b/python-clustering-api.appspot.com/o/images%2FFace%2F242.jpg?generation=1522585299229997&alt=media"
  ], 
  [
    0.7914929138145477, 
    "https://www.googleapis.com/download/storage/v1/b/python-clustering-api.appspot.com/o/images%2FFace%2F212.jpg?generation=1522584727478100&alt=media"
  ], 
  [
    0.7806927914191767, 
    "https://www.googleapis.com/download/storage/v1/b/python-clustering-api.appspot.com/o/images%2FFace%2F099.jpg?generation=1522585251917855&alt=media"
  ], 
  [
    0.6948463995381056, 
    "https://www.googleapis.com/download/storage/v1/b/python-clustering-api.appspot.com/o/images%2FFace%2F119.jpg?generation=1522584693369035&alt=media"
  ]
]

If the article helped you, please comment below. Feel free to ask if faced any problems reading the article.

Setting Up ArUco on your PC

Wed, 31 Jan 2018 00:00:00 +0000

Setting up ArUco Library on your PC

The ArUco library for OpenCV is a lightweight C++ wrapper for Augmented Reality applications. Its simple, efficient and versatile enough to detect hundreds of unique AR markers.

Pre Requisites

Setup OpenCV on your PC. Get the installation instructions here
Basic CMake knowledge. For starters, first two steps of the official documentation will be enough.

Installing ArUco Library

The latest release package can be found on sourceforge. Now you need to extract and compile the source code using cmake(the instructions mentioned below work on linux; for windows you can use use cmake-GUI).

Note that the official documentation recommends installing it in a working directory, rather than with sudo or admin privileges. Supposing you downloaded it in Downloads, open terminal — note that xxx is downloaded zip library version

cd ~/Downloads
unzip arucoxxx.zip
cd aurucoxxx
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=<pathToInstall>
make
make install

Here the pathToInstall will be preferred location. I kept it in the same folder as

-DCMAKE_INSTALL_PREFIX=../aruco_src

To make sure that ArUco finds the correct library files, generate a config file(refer this blog for more).

sudo gedit /etc/ld.so.conf.d/aruco.conf
# Add the following line and save it:
# /usr/local/lib

# Now, in your terminal, execute:
sudo ldconfig

Note that since the installation is not global you’ve to always include pathToInstall in CMakeLists.txt file of your project as

# Adding local ArUco Library
include_directories(/home/<pc-name>/Downloads/arucoxxx/aruco_src/include/)
link_directories(/home/<pc-name>/Downloads/arucoxxx/aruco_src/lib/)

That’s it. Found this article helpful? If so, please comment below. Feel free to ask any questions.

Introduction to ArUco

Wed, 31 Jan 2018 00:00:00 +0000

Building an AR Marker Detection program using ArUco Library

Introduction to AR using ArUco

While the current MR development revolves around hardware such as phones with depth sensors, Microsoft HoloLens, or HTC Vive, novices shouldn’t be undermined from the learning experience. Thanks to open source alternatives such as OpenCV and ArUco library, novices can develop AR apps using webcams for image input.

Pre Requisites

A desktop or laptop with webcam
Setup ArUco on device. Refer my previous post
Familiarity with OpenCV in C++. Refer the official website for tutorial Although I tried it on linux, windows users can google build commands

Why ArUco?

The official docs state,

ArUco: a minimal library for Augmented Reality applications based on OpenCV
Unlike recent libraries such as Unity, ARToolkit and SDK’s such as Google’s ARCore, ArUco offers easy setup and deployment. With just a few lines of code, one can develop an app for AR Marker detection and simple object rendering. Its suitable for beginning your AR journey — fast.

Jump Start to the code

Start the webcam : Create a camera capture object that queries the device for live video.

// Read the web cam
CvCapture *capture = 0;
capture = cvCreateCameraCapture( 0 );

Grab image from the webcam : OpenCV handles images in various formats. The simplest of all is a Mat object, storing images as matrices of their color values(RGB). Use the camera capture object to get the image from the webcam

//read the input image
IplImage *image = 0;
image = cvQueryFrame(capture);
// Gotta cast the image to Matrix
// Remember all computations are in Mat form,all visuals aren't so
cv::Mat InImage;
InImage = cv::cvarrToMat(image);

Detect and Highlight the markers : ArUco employs special markers for object detection and pose estimation. These are lightweight squares that carry bits of information. For quick build, you can use ArUco’s inbuilt function draw to highlight the marker. First parameter is the image, Scalar(0,0,255) decides the boundary outline color(green in this case) and last parameter is the boundary width.

//Ok, let's detect
MDetector.detect(InImage, Markers);
//for each marker, draw info and its boundaries in the image
for (auto &Marker : Markers) {
    cout << Marker << endl;
    Marker.draw(InImage, Scalar(0, 0, 255), 2);
}
The official repository provides a marker grid board in pdf format.

A 4x4 marker grid board. Credits : pokemonaaah.net

Display Loop : Finally, display the image and set the program in a loop so as detect markers in live video stream. To stop, you can add a break condition.

// start the loop
int key=0;
while(key != 'q') {
    // do the loop
    cv::imshow("in", InImage);
    key = cv::waitKey(1); //wait for key to be pressed
}

Building The Program using CMake

Note that unlike traditional programs, opencv programs (or C++ programs in general) require you to link source code to libraries you want to use. So you’ve to use tools such as CMake for a variety of purposes, namely

To generate object files or executables
To link source code to libraries
Maintain project details(metadata, version, author)
Include header files and other source code
You will use a simple CMake script to build your project now. I won’t explain it much(refer the official tutorial). Create a text file CMakeLists.txt in the current directory with contents :

cmake_minimum_required(VERSION 2.8)
project(helloAR)
SET(CMAKE_MODULE_PATH ${CMAKE_INSTALL_PREFIX}/lib/cmake/ )
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
# Required packages
find_package(OpenCV REQUIRED)
# Adding local ARUco Library
include_directories(pathToArUcoBuild/include/)
link_directories(pathToArUcoBuild/lib/)
add_executable(helloAR aruco_simple.cpp)
target_link_libraries(helloAR ${OpenCV_LIBS} aruco)

The key takeaway is to include and link directories appropriately. Now open terminal and start build process

cd working_directory
mkdir build
cd build
cmake ..
make
./helloAR

Result

Final marker detection result The build part may seem daunting at first(was for me). Using an IDE like Codeblocks or CLion will come handy as you develop more. For documentation, you can refer linux manpages (enter in terminalman cmake). You can get the full source code here.

That’s it. Were you able to develop this AR app? If yes, please leave a comment or if you have any questions.