Python decorators for dummies

If you’re going through interviews for the positions of Python developer, or looking forward to preparing for one, or just a curios developer, you better have your head clear around the concept of decorators in Python programming language.
I won’t be delving into ‘what are design patterns’ , and why should you make use of it, whenever possible. The post is merely about understanding and writing decorators in Python. You can find plethora of posts about Python decorators, the motivation for me is, that everyone has their own way of explaining, especially a technical concept.

Continue reading “Python decorators for dummies”

Deploy Django with NginX, Gunicorn, PostgreSQL, virtualenv

Step 0 – Update and upgrade

We are using Ubuntu 16.04 LTS for this tutorial
ubuntu version
apt-get update update the list of available packages and their versions, but it does not install or upgrade any packages. apt-get upgrade actually installs newer versions of the packages you have. After updating the lists, the package manager knows about available updates for the software you have installed.
$ sudo apt-get update
$ sudo apt-get upgrade

Continue reading “Deploy Django with NginX, Gunicorn, PostgreSQL, virtualenv”

Install Python 3.6 on Ubuntu 14.04 and 16.04 LTS

A regular Ubuntu release comes up with 9 months of support, except the LTS (Long Term Support) versions.  Ubuntu 14.04 and 16.04 being the LTS are still widely in use at production level. Being a Python developer the first thing I need to on having a fresh Ubuntu 14.04, or 16.04 machine is update Python. Ubuntu 14.04 has Python 3.4 and 16.04 comes with Python 3.5. This blog post is about installing Python 3.6 on your Ubuntu 14.04, or 16.04 LTS.

Continue reading “Install Python 3.6 on Ubuntu 14.04 and 16.04 LTS”

Apache Spark and PySpark on CentOS/RHEL 7.x

What is Apache Spark

You may have noticed, wherever there is a talk about big data the name Apache Spark eventually comes up, in simplest words it’s a large-scale data processing engine. Apache Spark is a fast data processing framework with provided APIs to connect and perform big data processing. Spark being the largest open-source data processing engine, has been adopted by large companies – Yahoo, eBay, Netflix, have massive scale Spark deployments, processing multiple petabytes of data on clusters of over 8,000 nodes.
Apache Spark can be started as a standalone cluster (which we’ll be doing for this tutorial), or using Mesos or YARN as cluster managers. Spark can work with data from various sources, AWS S3, HDFS, Cassandra, Hive (structured data), HBase, or any other Hadoop data source. Above all what makes Spark high in-demand is the included libraries MLib, SQL and DataFrames, GraphX, and Spark Streaming, to cater the main data processing use-cases, such that users can combinely use all these libraries in the same application.

Continue reading “Apache Spark and PySpark on CentOS/RHEL 7.x”

Install Jupyter Notebook

What is Jupyter Notebook

If you’re a Python developer, or someone who has to interact with Python, you may be hearing or seeing the term Jupyter Notebook quite lot, while reading articles, or looking for some solution on-line.
The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, machine learning and much more.

Continue reading “Install Jupyter Notebook”

Anaconda on CentOS 7.x

What is Anaconda

Anaconda is a free open-source Python distribution (as well as R programming language), intended for large-scale data processing and  analysis, and scientific computing. Anaconda Python  distribution is manged and developed by Continuum Analytics.
Anaconda (“Anaconda Distribution”) is a free, easy-to-install package manager, environment manager, Python distribution, and collection of over 720 open source packages with free community support. Hundreds more open source packages and their dependencies can be installed with a simple “conda install [packagename]”. It’s platform-agnostic, can be used on Windows, OS X and Linux. Or even easier.

Continue reading “Anaconda on CentOS 7.x”

conda VS pip

I decided to write this post, as I myself when for the first time tried to use conda (the package manager for Anaconda Python distribution, the first question was in what ways conda is better then pip, and so why one should think of preferring condo over the de-facto pip. Here I have put a comprehensive post about ‘getting started with conda’ i.e. what extra condo can offer.

A short comparison

pip
  • Can only be used for Python packages.
  • The supported package manger by the Python foundation, hence widely used.
conda
  • Handles library dependencies even outside Python i.e. packages for C libraries, or R packages, or really anything.
  • Supports virtual environment out of the box.
  • Developed to be used with Anaconda Python distribution, though can be used with the standard Python distribution – but highly not recommended.

Continue reading “conda VS pip”

Elasticsearch with Python

Start here:

Elasticsearch is a distributed storage and real-time search engine.
  • Distributed storage – you just need to setup and add Elasticsearch nodes, it’ll keep the data distributed on the cluster nodes. The distributed-ness makes data durable and highly-available too.
  • Real-time search engine – You can get to query the data the moment it’s been written.
Due to the above 2 attributes you have been listening and reading about Elasticsearch, wherever there’s a discussion of real-time data analysis. It’d not be an overstatement to say technologies like Elasticsearch set the foundation for any efficient and reliable search engine.
The ES itself is implemented in Java, but it provides a good RESTful api interface which makes it possible to use it with any programming language.

Continue reading “Elasticsearch with Python”

what’s the difference between pyenv, virtualenv, virtualenvwrapper

My first post was about Python ‘virtualenv‘ – started with what is a virtual environment, why we need it, and a minimal example.  The purpose was to clarify beginners about what they get to see in almost every Python example. After that I also wrote about pyenv, and virualenvwrapper, felt like I may have enhanced the confusion here, all the starters might be having the question – what’s the difference between pyenv, virtualenv, virtualenvwrapper ? hence the today’s post title.
As you all know already that a virtual environment is ” a separate Python interpreter with its own set of installed packages.”

Continue reading “what’s the difference between pyenv, virtualenv, virtualenvwrapper”