C++ is a powerful programming language that is often used for applications that require high performance and low-level system access. Python, on the other hand, is a popular high-level programming language that is easy to learn and has a vast library of modules and tools for scientific computing, data analysis, and web development. Sometimes, however, Python may not be able to provide the performance required for certain applications. Especially when dealing with complex computations or large datasets. In such cases, it is possible to use C++ code from Python to achieve the desired performance. In this article, we will explore how to use C++ from Python.
Why use C++ from Python?
There are many situations where you might want to use C++ code in a Python program. For example, you might have a C++ library that you want to use in your Python code, or you might want to speed up some computationally intensive code by writing it in C++.
You may have identified a performance bottleneck in your Python code, implementing the functionality in C++ could give you a significant performance boost.
Performance bottlenecks in Python programs can be rewritten in C++ for maximal speed.
While Python is known for its ease of use and readability, C++ is known for its speed and efficiency.
Fortunately, it is possible to use C++ code from within a Python program, allowing you to take advantage of the strengths of both languages.
Using C++ from Python can be an excellent solution to address the performance limitations of Python. This approach allows you to use the high-level features of Python while leveraging the speed of C++.
How to use C++ from Python?
One way to use C++ code from Python is to write a C++ library that can be called from Python. This involves creating a shared library (.so) or a dynamic-link library (.dll) file on Windows, that contains the C++ code.
To use C++ from Python, we use a technique called “Python C++ extensions”. This technique enables us to write C++ code and expose it as Python modules that can be imported and used in Python scripts.
Writing a C++ module for Python can be a powerful way to leverage the performance and efficiency of C++ code within a Python environment. There are several methods available, including using the Python/C API, using a C++ wrapper library like Boost.Python, using SWIG, using ctypes, and using Cython. The best method for your use case will depend on your requirements and the complexity of your C++ code.
Python/C API
The Python/C API (Application Programming Interface) is the official interface for writing C/C++ modules for Python. It provides a set of C functions that allow you to interact with Python objects and the Python interpreter. Using the Python/C API, you can write a C++ module that can be imported and used in Python code. When using the Python/C API, you write C++ modules for Python by defining functions in C++ and wrapping them in Python objects.
Here is an example of using the Python C API to create a simple C++ module for Python:
#include <Python.h>
static PyObject* greet(PyObject* self, PyObject* args) {
const char* name;
if (!PyArg_ParseTuple(args, "s", &name)) {
return NULL;
}
printf("Hello, %s!\n", name);
Py_RETURN_NONE;
}
static PyMethodDef module_methods[] = {
{"greet", greet, METH_VARARGS, "Greet someone."},
{NULL, NULL, 0, NULL}
};
static struct PyModuleDef module_definition = {
PyModuleDef_HEAD_INIT,
"example",
"A Python module written in C++",
-1,
module_methods
};
PyMODINIT_FUNC PyInit_example(void) {
return PyModule_Create(&module_definition);
}
In this example, we define a greet function that takes a string argument and prints a greeting to the console. We then define a PyMethodDef struct that describes our module’s methods, including our greet function. Finally, we define a PyModuleDef struct that describes our module and its methods, and we define a PyInit_example function that initializes our module.
ctypes
ctypes is a foreign function library for Python. It provides C compatible data types, and allows calling functions in DLLs or shared libraries written in C or C++. It can be used to wrap these libraries in pure Python. This approach requires that you compile your C++ code into a shared library (.so or .dll file), which can then be loaded into Python using ctypes. Using ctypes is easy and is suitable for simple C++ libraries with a C-compatible API.
Here’s an example of how to use ctypes to call a C++ function from Python:
C++:
extern "C" {
int add(int a, int b) {
return a + b;
}
}
To use this C++ function from Python, we need to compile it into a shared library. This step requires a C++ compiler and the appropriate build tools.
For example, suppose you are using the GCC compiler on a Linux system. In this case, you can build the shared library with:
g++ -shared -o libsum.so -fPIC sum.cpp
The -fPIC option tells GCC to generate position-independent code (PIC) which is required for most architectures (it’s not vital on x86, but it’s still a good idea as it allows code pages from the library to be shared between processes).
Next, we can load the library and call the sum function from Python using ctypes:
Python:
# sum.py
import ctypes
# Load the shared library
lib = ctypes.CDLL('./libsum.so')
# Define the argument and return types of the sum function
lib.sum.argtypes = (ctypes.c_int, ctypes.c_int)
lib.sum.restype = ctypes.c_int
# Call the sum function
result = lib.sum(1, 2)
# Print the result
print(result) # 3`
When we run this Python script, we should see the sum of 1 and 2 printed to the console.
As an example of the performance improvement achievable by using ctypes C++ Extensions from Python, here is a C++ class with a member function countBig() that loops 10,000,000 times:
#include <iostream>
class HelloWorld {
public:
void sayHello() const {
const std::string message = "Hello from C++";
std::cout << message << std::endl;
}
void countBig() const {
int a;
const int loopLimit = 10000000;
for (int n = 0; n < loopLimit; ++n) {
a = 1;
}
}
};
int main() {
HelloWorld hello;
hello.sayHello();
return 0;
}
extern "C" {
HelloWorld* hello_new() { return new HelloWorld(); }
void HelloWorld_sayHello(const HelloWorld* hello) { hello->sayHello(); }
void HelloWorld_countBig(const HelloWorld* hello) { hello->countBig(); }
}
Python:
from ctypes import cdll
import timeit
import os
def runme():
# Load
lib = cdll.LoadLibrary('./libhello.so')
obj = lib.hello_new()
lib.HelloWorld_sayHello(obj)
# python loop
start = timeit.default_timer()
for i in range(1, 10000000):
a = 1
stop = timeit.default_timer()
print("python: ", stop - start)
# c++ loop
start = timeit.default_timer()
lib.HelloWorld_countBig(obj)
stop = timeit.default_timer()
print("c++: ", stop - start)
runme()
Result:
Hello from C++
('python: ', 0.5580978393554688)
('c++: ', 0.013834953308105469)
In this case the C++ loop is more than 39 times faster than Python!
SWIG
The Simplified Wrapper and Interface Generator (SWIG) is a tool that automates the process of creating Python bindings for C++ code. SWIG generates a Python module that provides Python wrappers around the C++ code, making it easy to call the C++ functions from Python. SWIG supports a wide range of C++ features, including classes, namespaces, and templates. Using SWIG can be more complex than using ctypes, but it provides more control over the generated Python interface.
Here’s an example of how to use SWIG to generate Python bindings for a C++ library:
C++:
/* File : example.cpp */
/* A global variable */
double Foo = 3.0;
/* Compute the greatest common divisor of positive integers */
int gcd(int x, int y) {
int g;
g = y;
while (x > 0) {
g = x;
x = y % x;
y = g;
}
return g;
}
SWIG interface file:
/* File : example.i */
%module example
%inline %{
extern int gcd(int x, int y);
extern double Foo;
%}
Once you have created the SWIG interface file, you can use SWIG to generate the Python module:
$ swig -c++ -python example.i
$ g++ -c example.cpp example_wrap.cxx -I/usr/include/python3.5
$ g++ -shared example.o example_wrap.o -o _example.so`
After generating the Python module, you can use it from Python like this:
Python:
import example
# Call our gcd() function
x = 42
y = 105
g = example.gcd(x, y)
print("The gcd of %d and %d is %d" % (x, y, g))
# Manipulate the Foo global variable
# Output its current value
print("Foo = %s" % example.cvar.Foo)
# Change its value
example.cvar.Foo = 3.1415926
# See if the change took effect
print("Foo = %s" % example.cvar.Foo)
Boost.Python
Another way to use C++ code from Python is to use the Boost.Python library. It is a C++ library that allows us to expose C++ classes and functions to Python. Like SWIG, Boost.Python generates a Python module that provides Python wrappers around the C++ code. This library provides a higher-level interface for integrating C++ code with Python. It simplifies the process of writing C++ modules for Python by providing abstractions that handle the details of the Python/C API for you.
Using Boost.Python provides flexibility and allows us to create modules that are more similar to native Python modules. It also provides advanced features such as automatic conversion between C++ and Python types.
To use Boost.Python, we need to create a C++ module that defines the classes and functions we want to expose to Python. Here’s an example:
// sum.cpp
#include <boost/python.hpp>
int sum(int a, int b) {
return a + b;
}
BOOST_PYTHON_MODULE(sum) {
using namespace boost::python;
def("sum", sum);
}
In this example, we define the sum function and then use the BOOST_PYTHON_MODULE macro to create a Python module that exposes the sum function to Python. We also use the boost::python namespace to define the def function, which creates a Python function object that calls the sum function.
To use this C++ module from Python, we can import it like any other Python module:
Python:
# sum.py
import sum
result = sum.sum(1, 2)
print(result)`
When we run this Python script, we should see the sum of 1 and 2 printed to the console.
Cython
Cython is a superset of Python that allows you to write Python code that can be compiled to C or C++ code. It provides a way to write Python code that is compiled to high-performance C or C++ code, which can then be used as a Python module. This can be a useful approach if you already have Python code that you want to optimize with C++.
Here is an example of using Cython to create a simple C++ module for Python:
First, we create a file called example.hpp and add the following code to define a simple C++ function:
C++:
#ifndef EXAMPLE_HPP
#define EXAMPLE_HPP
int add_numbers(int a, int b);
#endif
Then, we create a file called example.cpp and add the following code to implement the function defined in example.hpp:
#include "example.hpp"
int add_numbers(int a, int b) {
return a + b;
}
Next, we create a file called example.pyx and add the following code to define a Python interface for the add_numbers function using Cython:
Cython:
cdef extern from "example.hpp":
int add_numbers(int a, int b)
def py_add_numbers(int a, int b):
return add_numbers(a, b)
Finally, create a setup.py file to build the Cython module:
Python:
from distutils.core import setup
from Cython.Build import cythonize
setup(
ext_modules=cythonize("example.pyx"),
language="c++"
)
This code uses the distutils module to compile the “example.pyx” file into a C++ module that can be imported into Python. The “language” parameter tells Cython to generate C++ code instead of C code.
The module is built by running the following command in your terminal:
python setup.py build_ext --inplace
Once the module is built, you can import it in Python and use the py_add_numbers function:
Python:
import example
result = example.py_add_numbers(1, 2)
print(result) # Output: 3`
Conclusion
Using C++ extension modules for Python can be a powerful way to achieve high performance and extend the functionality of your Python applications.
There are several methods for creating C++ modules for Python, including the Python/C API, ctypes, Boost.Python, SWIG and Cython. Each method has its own benefits and drawbacks, so it’s important to choose the method that best fits your needs and skills.
Contact us to get help building C++ extension modules for your Python application.
