1. Expanding Your Python Knowledge: LesserKnown Libraries. . . . . . . 1 The Standard Library 2 In the Wild 15 Easier Python Packaging with flit 16 CommandLine Applications 19 Graphical User Interfaces 26 System Tools 34 Web APIs with hug 41 Dates and Times 46 GeneralPurpose Libraries 53 Conclusion 66 v CHAPTER 1 Expanding Your Python Knowledge: LesserKnown Libraries The Python ecosystem is vast and farreaching in both scope and depth. Starting out in this crazy, opensource forest is daunting, and even with years of experience, it still requires continual effort to keep uptodate with the best libraries and techniques. In this report we take a look at some of the lesserknown Python libraries and tools. Python itself already includes a huge number of highquality libraries; collectively these are called the standard library. The standard library receives a lot of attention, but there are still some libraries within it that should be better known. We will start out by discussing several, extremely useful tools in the standard library that you may not know about. We’re also going to discuss several exciting, lesserknown libraries from the thirdparty ecosystem. Many highquality thirdparty libraries are already wellknown, including Numpy and Scipy, Django, Flask, and Requests; you can easily learn more about these libraries by searching for information online. Rather than focusing on those standouts, this report is instead going to focus on several interesting libraries that are growing in popularity. Let’s start by taking a look at the standard library. 1 1 Looking for sorted container types? The excellent sorted containers package has highperformance sorted versions of the list, dict, and set datatypes. The Standard Library The libraries that tend to get all the attention are the ones heavily used for operatingsystem interaction, like sys, os, shutil, and to a slightly lesser extent, glob. This is understandable because most Python applications deal with input processing; however, the Python standard library is very rich and includes a bunch of additional functionality that many Python programmers take too long to dis‐ cover. In this chapter we will mention a few libraries that every Python programmer should know very well. collections First up we have the collections module. If you’ve been working with Python for any length of time, it is very likely that you have made use of the this module; however, the batteries contained within are so important that we’ll go over them anyway, just in case. collections.OrderedDict collections.OrderedDict gives you a dict that will preserve the order in which items are added to it; note that this is not the same as a sorted order.1 The need for an ordered dict comes up surprisingly often. A com‐ mon example is processing lines in a file where the lines (or some‐ thing within them) maps to other data. A mapping is the right solution, and you often need to produce results in the same order in which the input data appeared. Here is a simple example of how the ordering changes with a normal dict: >>> dict(zip(ascii_lowercase, range(4))) {a: 0, b: 1, c: 2, d: 3} >>> dict(zip(ascii_lowercase, range(5))) {a: 0, b: 1, c: 2, d: 3, e: 4} >>> dict(zip(ascii_lowercase, range(6))) {a: 0, b: 1, c: 2, d: 3, f: 5, e: 4} 2 | Chapter 1: Expanding Your Python Knowledge: LesserKnown Libraries >>> dict(zip(ascii_lowercase, range(7))) {a: 0, b: 1, c: 2, d: 3, g: 6, f: 5, e: 4} See how the key f now appears before the e key in the sequence of keys? They no longer appear in the order of inser‐ tion, due to how the dict internals manage the assignment of hash entries. The OrderedDict, however, retains the order in which items are inserted: >>> from collections import OrderedDict >>> OrderedDict(zip(ascii_lowercase, range(5))) OrderedDict((a, 0), (b, 1), (c, 2), (d, 3), (e, 4)) >>> OrderedDict(zip(ascii_lowercase, range(6))) OrderedDict((a, 0), (b, 1), (c, 2), (d, 3), (e, 4), (f, 5)) >>> OrderedDict(zip(ascii_lowercase, range(7))) OrderedDict((a, 0), (b, 1), (c, 2), (d, 3), (e, 4), (f, 5), (g, 6)) OrderedDict: Beware creation with keyword arguments There is an unfortunate catch with OrderedDict you need to be aware of: it doesn’t work when you create the OrderedDict with keyword arguments, a very common Python idiom: >>> collections.OrderedDict(a=1,b=2,c=3) OrderedDict((b, 2), (a, 1), (c, 3)) This seems like a bug, but as explained in the docu‐ mentation, it happens because the keyword arguments are first processed as a normal dict before they are passed on to the OrderedDict. collections.defaultdict collections.defaultdict is another specialcase dictionary: it allows you to specify a default value for all new keys.
20 Python Libraries You Aren’t Using (But Should) Caleb Hattingh Beijing Boston Farnham Sebastopol Tokyo 20 Python Libraries You Aren’t Using (But Should) by Caleb Hattingh Copyright © 2016 O’Reilly Media Inc All rights reserved Printed in the United States of America Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472 O’Reilly books may be purchased for educational, business, or sales promotional use Online editions are also available for most titles (http://safaribooksonline.com) For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com Editor: Dawn Schanafelt Production Editor: Colleen Lobner Copyeditor: Christina Edwards August 2016: Interior Designer: David Futato Cover Designer: Randy Comer Illustrator: Rebecca Demarest First Edition Revision History for the First Edition 2016-08-08: First Release The O’Reilly logo is a registered trademark of O’Reilly Media, Inc 20 Python Libra‐ ries You Aren’t Using (But Should), the cover image, and related trade dress are trade‐ marks of O’Reilly Media, Inc While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limi‐ tation responsibility for damages resulting from the use of or reliance on this work Use of the information and instructions contained in this work is at your own risk If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsi‐ bility to ensure that your use thereof complies with such licenses and/or rights 978-1-491-96792-8 [LSI] Table of Contents Expanding Your Python Knowledge: Lesser-Known Libraries The Standard Library In the Wild Easier Python Packaging with flit Command-Line Applications Graphical User Interfaces System Tools Web APIs with hug Dates and Times General-Purpose Libraries Conclusion 15 16 19 26 34 41 46 53 66 v CHAPTER Expanding Your Python Knowledge: Lesser-Known Libraries The Python ecosystem is vast and far-reaching in both scope and depth Starting out in this crazy, open-source forest is daunting, and even with years of experience, it still requires continual effort to keep up-to-date with the best libraries and techniques In this report we take a look at some of the lesser-known Python libraries and tools Python itself already includes a huge number of high-quality libraries; collectively these are called the standard library The standard library receives a lot of attention, but there are still some libraries within it that should be better known We will start out by discussing several, extremely useful tools in the standard library that you may not know about We’re also going to discuss several exciting, lesser-known libraries from the third-party ecosystem Many high-quality third-party libraries are already well-known, including Numpy and Scipy, Django, Flask, and Requests; you can easily learn more about these libraries by searching for information online Rather than focusing on those standouts, this report is instead going to focus on several interesting libraries that are growing in popularity Let’s start by taking a look at the standard library The Standard Library The libraries that tend to get all the attention are the ones heavily used for operating-system interaction, like sys, os, shutil, and to a slightly lesser extent, glob This is understandable because most Python applications deal with input processing; however, the Python standard library is very rich and includes a bunch of additional functionality that many Python programmers take too long to dis‐ cover In this chapter we will mention a few libraries that every Python programmer should know very well collections First up we have the collections module If you’ve been working with Python for any length of time, it is very likely that you have made use of the this module; however, the batteries contained within are so important that we’ll go over them anyway, just in case collections.OrderedDict collections.OrderedDict gives you a dict that will preserve the order in which items are added to it; note that this is not the same as a sorted order.1 The need for an ordered dict comes up surprisingly often A com‐ mon example is processing lines in a file where the lines (or some‐ thing within them) maps to other data A mapping is the right solution, and you often need to produce results in the same order in which the input data appeared Here is a simple example of how the ordering changes with a normal dict: >>> dict(zip(ascii_lowercase, range(4))) {'a': 0, 'b': 1, 'c': 2, 'd': 3} >>> dict(zip(ascii_lowercase, range(5))) {'a': 0, 'b': 1, 'c': 2, 'd': 3, 'e': 4} >>> dict(zip(ascii_lowercase, range(6))) {'a': 0, 'b': 1, 'c': 2, 'd': 3, 'f': 5, 'e': 4} Looking for sorted container types? The excellent sorted containers package has high- performance sorted versions of the list, dict, and set datatypes | Chapter 1: Expanding Your Python Knowledge: Lesser-Known Libraries >>> dict(zip(ascii_lowercase, range(7))) {'a': 0, 'b': 1, 'c': 2, 'd': 3, 'g': 6, 'f': 5, 'e': 4} See how the key "f" now appears before the "e" key in the sequence of keys? They no longer appear in the order of inser‐ tion, due to how the dict internals manage the assignment of hash entries The OrderedDict, however, retains the order in which items are inserted: >>> from collections import OrderedDict >>> OrderedDict(zip(ascii_lowercase, range(5))) OrderedDict([('a', 0), ('b', 1), ('c', 2), ('d', 3), ('e', 4)]) >>> OrderedDict(zip(ascii_lowercase, range(6))) OrderedDict([('a', 0), ('b', 1), ('c', 2), ('d', 3), ('e', 4), ('f', 5)]) >>> OrderedDict(zip(ascii_lowercase, range(7))) OrderedDict([('a', 0), ('b', 1), ('c', 2), ('d', 3), ('e', 4), ('f', 5), ('g', 6)]) OrderedDict: Beware creation with keyword arguments There is an unfortunate catch with OrderedDict you need to be aware of: it doesn’t work when you create the OrderedDict with keyword arguments, a very common Python idiom: >>> collections.OrderedDict(a=1,b=2,c=3) OrderedDict([('b', 2), ('a', 1), ('c', 3)]) This seems like a bug, but as explained in the docu‐ mentation, it happens because the keyword arguments are first processed as a normal dict before they are passed on to the OrderedDict collections.defaultdict collections.defaultdict is another special-case dictionary: it allows you to specify a default value for all new keys The Standard Library | Here’s a common example: >>> d = collections.defaultdict(list) >>> d['a'] [] You didn’t create this item yet? No problem! Key lookups auto‐ matically create values using the function provided when creat‐ ing the defaultdict instance By setting up the default value as the list constructor in the preced‐ ing example, you can avoid wordy code that looks like this: d = {} for k in keydata: if not k in d: d[k] = [] d[k].append( ) The setdefault() method of a dict can be used in a somewhat similar way to initialize items with defaults, but defaultdict gener‐ ally results in clearer code.2 In the preceding examples, we’re saying that every new element, by default, will be an empty list If, instead, you wanted every new ele‐ ment to contain a dictionary, you might say defaultdict(dict) collections.namedtuple The next tool, collections.namedtuple, is magic in a bottle! Instead of working with this: tup = (1, True, "red") You get to work with this: >>> from collections import namedtuple >>> A = namedtuple('A', 'count enabled color') >>> tup = A(count=1, enabled=True, color="red") >>> tup.count >>> tup.enabled True For instance, this example with setdefault() looks like d.setdefault(k, []).append( ) The default value is always evaluated, whereas with defaultdict the default value generator is only evaluated when necessary But there are still cases where you’ll need setdefault(), such as when using different default values depending on the key | Chapter 1: Expanding Your Python Knowledge: Lesser-Known Libraries ... 20 Python Libraries You Aren’t Using (But Should) Caleb Hattingh Beijing Boston Farnham Sebastopol Tokyo 20 Python Libraries You Aren’t Using (But Should) by Caleb Hattingh Copyright © 201 6... Release The O’Reilly logo is a registered trademark of O’Reilly Media, Inc 20 Python Libra‐ ries You Aren’t Using (But Should), the cover image, and related trade dress are trade‐ marks of O’Reilly... logging.basicConfig(level=args.loglevel) With this setup, if you call your program with 12 | Chapter 1: Expanding Your Python Knowledge: Lesser-Known Libraries $ python main.py -ll DEBUG it will run with the