Practical Business Python

Taking care of business, one python script at a time

Mon 20 January 2020

Using Markdown to Create Responsive HTML Emails

Posted by Chris Moffitt in articles   

article header image

Introduction

As part of managing the PB Python newsletter, I wanted to develop a simple way to write emails once using plain text and turn them into responsive HTML emails for the newsletter. In addition, I needed to maintain a static archive page on the blog that links to the content of each newsletter. This article shows how to use python tools to transform a markdown file into a responsive HTML email suitable for a newsletter as well as a standalone page integrated into a pelican blog.

Rationale

I am a firm believer in having access to all of the content I create in a simple text format. That is part of the reason why I use pelican for the blog and write all content in restructured text. I also believe in hosting the blog using static HTML so it is fast for readers and simple to distribute. Since I spend a lot of time creating content, I want to make sure I can easily transform it into another format if needed. Plain text files are the best format for my needs.

As I wrote in my previous post, Mailchimp was getting cost prohibitive. In addition, I did not like playing around with formatting emails. I want to focus on content and turning it into a clean and responsive email - not working with an online email editor. I also want the newsletter archives available for people to view and search in a more integrated way with the blog.

One thing that Mailchimp does well is that it provides an archive of emails and ability for the owner to download them in raw text. However, once you cancel your account, those archives will go away. It’s also not very search engine friendly so it’s hard to reference back to it and expose the content to others not subscribed to the newsletter.

With all that in mind, here is the high level process I had in mind:

Markdown email flow

HTML Email

Before I go through the python scripts, here’s some background on developing responsive HTML-based emails. Unfortunately, building a template that works well in all email clients is not easy. I naively assumed that the tips and tricks that work for a web site would work in an HTML email. Unfortunately that is not the case. The best information I could find is that you need to use HTML tables to format messages so they will look acceptable in all the email clients. Yuck. I feel like I’m back in Geocities.

Geocities

This is one of the benefits that email vendors like Mailchimp provide. They will go through all the hard work of figuring out how to make templates that look good everywhere. For some this makes complete sense. For my simple needs, it was overkill. Your mileage may vary.

Along the way, I found several resources that I leveraged for portions of my final solution. Here they are for reference:

Besides having to use HTML tables, I learned that it is recommended that all the CSS be inlined in the email. In other words, the email needs to have all the styling included in the tags using style :

<h2 style='color:#337ab7; font-family:"Fjalla One", sans-serif; font-weight:500; margin:0; font-size:125%'>
    Other news
</h2>

Once again this is very old school web and would be really painful if not for tools that will do the inlining for you. I used the excellent premailer library to take an embedded CSS stylesheet and inline with the rest of the HTML.

You can find a full HTML template and all the code on github but here is a simple summary for reference. Please use the github version since this one is severely simplified and likely won’t work as is:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title>{{title}}</title>
<style type="text/css">
    body {
    margin: 0 !important;
    padding: 0 !important;
    width: 100% !important;
    color: #333;
    font-family: 'Average Sans', sans-serif;
    font-size: 14px;
    }
</style>
</head>
<body>
<center>
    <div style="background-color:#F2F2F2; max-width: 640px; margin: auto;">
    <table width="640" cellspacing="0" cellpadding="0" border="0" align="center" style="max-width:640px; width:100%;" bgcolor="#FFFFFF">
        <tr>
        <td align="center" valign="top" style="padding:10px;">
            <table width="600" cellspacing="0" cellpadding="0" border="0" align="center"
            style="max-width:600px; width:100%;">
            <tr>
                <td align="left" valign="top" style="padding:10px;">
                {{email_content}}
                </td>
            </tr>
    </table>
    </div>
    <p
    style="border-top: 1px solid #c6c6c6; color: #a9a9a9; margin-top: 50px; padding-top: 20px;font-size:13px; margin-bottom: 13px;">
    You received this email because you subscribed to our list.
        You can  <a href="{{UnsubscribeURL}}" style="color:#a9a9a9;" target="_blank" data-premailer="ignore">unsubscribe</a> at any time.</p>
        <p style="color: #a9a9a9;margin-bottom: 13px;font-size:13px;">{{SenderInfoLine}}</p>
</center>
</body>
</html>

This is a jinja template and you will notice that there is a place for email_content and title . The next step in the process is to render a markdown text file into HTML and place that HTML snippet into a template.

Markdown Article

Now that we know how we want the HTML to look, let’s create a markdown file. The only twist with this solution is that I want to create one markdown file that can be rendered in pelican and used for the HTML email.

Here is what a simple markdown file( sample_doc.md ) looks like that will work with pelican:

Title: Newsletter Number 6
Date: 12-9-2019 10:04am
Template: newsletter
URL: newsletter/issue-6.html
save_as: newsletter/issue-6.html

Welcome to the 6th edition of this newsletter.

## Around the site

* [Combining Multiple Excel Worksheets Into a Single Pandas Dataframe](https://pbpython.com/pandas-excel-tabs.html)
covers a simple approach to parse multiple excel tabs into one DataFrame.

## Other news

* [Altair](https://altair-viz.github.io/index.html) just released a new version. If you haven't looked at it in a while,
check out some of the [examples](https://altair-viz.github.io/gallery/index.html) for a snapshot of what you can do with it.

## Final Words

Thanks again for subscribing to the newsletter. Feel free to forward it on to others that may be interested.

The required input file uses standard markdown. The one tricky aspect is that the top 5 lines contain meta-data that pelican needs to make sure the correct url and templates are used when creating the output. Our final script will need to remove them so that it does not get rendered into the newsletter email. If you are not trying to incorporate into your blog, you can remove these lines.

If you are interested in incorporating this in your pelican blog, here is how my content is structured:

content
├── articles
├── extras
├── images
├── news
├── newsletter
│   ├── number_1.md
│   ├── number_2.md
│   ├── number_3.md
│   ├── number_4.md
│   ├── number_5.md
│   └── number_6.md
└── pages

All of the newsletter markdown files are stored in the newsletter directory and the blog posts are stored in the articles directory.

The final configuration I had to make in the pelicanconf.py file was to make sure the paths were setup correctly:

PATH = 'content'
PAGE_PATHS = ['newsletter', 'pages', 'news']

Now the blog is properly configured to render one of the newsletters.

Python code

Now that we have HTML template and the markdown document, we need a short python script to pull it all together. I will be using the following libraries so make sure they are all installed:

Additionally, make sure you are using python3 so you have access to pathlib and argparse .

In order to keep the article compact, I am only including the key components. Please look at the github repo for an approach that is a proper python standalone program that can take arguments from the command line.

The first step, import everything:

from markdown2 import Markdown
from pathlib import Path
from jinja2 import Environment, FileSystemLoader
from premailer import transform
from argparse import ArgumentParser
from bs4 import BeautifulSoup

Setup the input files and output HTML file:

in_doc = Path.cwd() / 'sample_doc.md'
template_file = 'template.html'
out_file = Path.cwd() / f'{in_doc.stem}_email.html'

Please refer to the pathlib article if you are not familiar with how or why to use it.

Now that the files are established, we need to read in the markdown file and parse out the header meta-data:

with open(in_doc) as f:
    all_content = f.readlines()

Using readlines to read the file ensures that each line in the file is stored in a list. This approach works for our small file but could be problematic if you had a massive file that you did not want to read into memory at once. For an email newsletter you should be ok with using readlines .

Here is what it all_content[0:6] looks like:

['Title: Newsletter Number 6\n',
'Date: 12-9-2019 10:04am\n',
'Template: newsletter\n',
'URL: newsletter/issue-6.html\n',
'save_as: newsletter/issue-6.html\n',
'\n']

We can clean up the title line for insertion into the template:

title_line = all_content[0]
title = f'PB Python - {title_line[7:].strip()}'

Which renders a title PB Python - Newsletter Number 6

The final parsing step is to get the body into a single list without the header:

body_content = all_content[6:]

Convert the raw markdown into a simple HTML string:

markdowner = Markdown()
markdown_content = markdowner.convert(''.join(body_content))

Now that the HTML is ready, we need to insert it into our jinja template:

# Set up jinja templates
env = Environment(loader=FileSystemLoader('.'))
template = env.get_template(template_file)
template_vars = {'email_content': markdown_content, 'title': title}
raw_html = template.render(template_vars)

At this point, raw_html has a fully formed HTML version of the newsletter. We need to use premailer’s transform to get the CSS inlined. I am also using BeautifulSoup to do some cleaning up and formatting of the HTML. This is purely aesthetic but I think it’s simple enough to do so I am including it:

soup = BeautifulSoup(transform(raw_html),
                    'html.parser').prettify(formatter="html")

The final step is to make sure that the unsubscribe link does not get mangled. Depending on your email provider, you may not need to do this:

final_HTML = str(soup).replace('%7B%7BUnsubscribeURL%7D%7D',
                            '{{UnsubscribeURL}}')
out_file.write_text(final_HTML)

Here is an example of the final email file:

Final newsletter

You should be able to copy and paste the raw HTML into your email marketing campaign and be good to go. In addition, this file will render properly in pelican. See this page for some past examples.

Summary

Markdown is a simple text format that can be parsed and turned into HTML using various python tools. In this case, the markdown file can be combined with a responsive HTML email template to simplify the process of generating content for newsletters. The added bonus is that the content can be included in a static blog so that it is searchable and easily available to your readers.

This solution is not limited to just building emails. Now that newer versions of pandas will include a native to_markdown method, this general approach could be extended to other uses. Using these principles you can build fairly robust reports and documents using markdown then incorporate the dataframe output into the final results. If there is interest in an example, let me know in the comments.

Comments