This guest post that walks through a great example of using python to automate a report generating process. I think PB Python readers will enjoy learning from this real world example using python, jupyter notebooks, papermill and several other tools.
I have been working on a side project so I have not had as much time to blog. Hopefully I will be able to share more about that project soon.
In the meantime, I wanted to write an article about styling output in pandas. The API for styling is somewhat new and has been under very active development. It contains a useful set of tools for styling the output of your pandas DataFrames and Series. In my own usage, I tend to only use a small subset of the available options but I always seem to forget the details. This article will show examples of how to format numbers in a pandas DataFrame and use some of the more advanced pandas styling visualization options to improve your ability to analyze data with pandas.
There are many sophisticated models people can build for solving a forecasting problem. However, they frequently stick to simple Excel models based on average historical values, intuition and some high level domain-specific heuristics. This approach may be precise enough for the problem at hand but there are alternatives that can add more information to the prediction with a reasonable amount of additional effort.
One approach that can produce a better understanding of the range of potential outcomes and help avoid the “flaw of averages” is a Monte Carlo simulation. The rest of this article will describe how to use python with pandas and numpy to build a Monte Carlo simulation to predict the range of potential values for a sales compensation budget. This approach is meant to be simple enough that it can be used for other problems you might encounter but also powerful enough to provide insights that a basic “gut-feel” model can not provide on its own.
Several years ago, I wrote an article about using pandas to creating a diff of two excel files. Ovet the years, the pandas API has changed and the diff script no longer works with the latest pandas releases. Through the magic of search engines, people are still discovering the article and are asking for help in getting it to work with more recent versions of pandas. Since pandas is closing in on a 1.0 release, I think this is a good time to get an updated version out there.
Over the past couple of months, there has been an ongoing discussion about Jupyter Notebooks affectionately called the “Notebook Wars”. The genesis of the discussion is Joel Grus’ presentation I Don’t Like Notebooks and has been followed up with Tim Hopper’s response, aptly titled I Like Notebooks. There have been several follow-on posts on this topic including thoughtful analysis from Yihui Xie.
The purpose of this post is to use some of the points brought up in these discussions as a background for describing my personal best practices for the analysis I frequently perform with notebooks. In addition, this approach can be tailored for your unique situation. I think many new python users do not take the time to think through some of these items I discuss. My hope is that this article will spark some discussion and provide a framework that others can build off for making repeatable and easy to understand data analysis pipelines that fit their needs.
Pandas offers several options for grouping and summarizing data but this variety of
options can be a blessing and a curse. These approaches are all powerful data
analysis tools but it can be confusing to know whether to use a
crosstab to build a summary table.
Since I have previously covered pivot_tables, this article will discuss the
crosstab function, explain its usage and illustrate how it can be
used to quickly summarize data. My goal is to have this article be a resource that
you can bookmark and refer to when you need to remind yourself what you can do
Python has many options for natively creating common Microsoft Office file types including Excel, Word and PowerPoint. In some cases, however, it may be too difficult to use the pure python approach to solve a problem. Fortunately, python has the “Python for Windows Extensions” package known as pywin32 that allows us to easily access Window’s Component Object Model (COM) and control Microsoft applications via python. This article will cover some basic use cases for this type of automation and how to get up and running with some useful scripts.
This article is a review of Chris Albon’s book, Machine Learning with Python Cookbook. This book is in the tradition of other O’Reilly “cookbook” series in that it contains short “recipes” for dealing with common machine learning scenarios in python. It covers the full spectrum of tasks from simple data wrangling and pre-processing to more complex machine learning model development and deep learning implementations. Since this is such a fast moving and broad topic, it is nice to get a new book that covers the latest topics and presents them in a compact but very useful format. Bottom line, I enjoyed reading this book and think it will be a useful resource to have on my python bookshelf. Read on for some more details about the book and who will benefit most from reading it.
This brief article introduces a flowchart that shows how to select a python visualization tool for the job at hand. The criteria for choosing the tools is weighted more towards the “common” tools out there that have been in use for several years. There may be some debate about some of the recommendations but I believe this should be helpful for someone that is new to the python visualization landscape and trying to make a decision about where to invest their time to learn how to use one of these libraries.