Python has been a popular programming language for more than 30 years, used by individuals and companies in a variety of industries. Developers love it for its clean syntax, clear documentation, and abundant libraries. Referring to the Stack Overflow 2022 Developer Survey, Python enters the list of the top three most popular languages among beginner developers.
However, there are times when the code written in Python is not as performant or secure as it could be. Rust is a more modern language that offers better protection from memory corruption, concurrency features, and security benefits compared to Python.
This blog post explores how translating Python code to Rust can give it more appropriate characteristics for a production environment. If you’re interested in a practical guide on Rust web development, you can check out this Yalantis’s article. But now let’s dive into the Python-Rust comparison.
What is Rust used for? Python projects that can run better in Rust
Python is essentially an interpreted language, and its main implementation in the CPython interpreter isn’t very fast. This is related to both the global interpreter lock (GIL) and the memory system (all objects take up a lot of memory and take longer to operate).
Many optimization and acceleration solutions have been created over time. For example, you can compile ready-made code in Cython (a superset of Python that provides C-like performance), or use Cython more widely and write a module on it, but writing modules on Cython can be rather difficult.
A much wiser thing to do would be to rewrite part of the Python code in Rust, which will bring increased speed and provide additional opportunities for working with streams and processes. Rust implements the principle of fearless concurrency, ensuring safe work with threads and processes. That’s why Rust and Python play well in data processing: Python gives a nice interface to work with, and Rust takes the first step toward high data processing speed.
Talking about web development, then in general, if you want to rewrite the entire service, it is customary to use Golang because the language is quite simple and understandable for pythonists after a few weeks. But Golang also has a GIL, and for real-time services, it can become a problem as the system will stop periodically to release resources. Since there is no such problem in Rust, it can be perfect for web services where real-time is important.
Also, Python is often used to build command line (CLI) applications. Mainly because it’s fast and convenient. But if the CLI is cross-platform (and we don’t want to require the client to install Python), then we have to bother with packaging. If there is a requirement to hide the code, then you will have to bother with obfuscators, which still don’t give the desired result. And when there are issues with the speed of individual parts, you still have to use something like Cython for optimization, and there is also compilation for a different platform, which complicates the assembly process. That’s why sometimes it is easier to write immediately in Rust and not bother with Python code workarounds.
Importantly, you’ll need to set your priorities right and consider using Rust instead of Python only if there are persuasive reasons for that.
First steps in using Rust with Python
We recommend a gradual start with Rust for Python programmers. It isn’t necessary to rewrite the whole system or application you’re working on in Rust, you can first start integrating Rust in the parts where high performance and security are of the highest priority. Such solutions as PyO3 help to combine Python and Rust code in a single project without much effort.
To use this service, you’ll need to install Rust (minimum version 1.48) and Python (minimum version 3.7), set up a virtual environment, and install a package manager (for PyO3, maturin is recommended).
Here is how to prepare Rust code to be available in the Python project using PyO3:
#![allow(unused)]
fn main() {
use pyo3::prelude::*;
/// Formats the sum of two numbers as string.
#[pyfunction]
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
Ok((a + b).to_string())
}
/// A Python module implemented in Rust. The name of this function must match
/// the `lib.name` setting in the `Cargo.toml`, else Python will not be able to
/// import the module.
#[pymodule]
fn pyo3_example(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;
Ok(())
}
}
*code snippet example is taken from PyO3 user guide
Example of transitioning from Python to Rust
Rust has notable advantages in concurrent and multi-threaded applications. At the level of the type system, Rust helps to avoid problems related to data races and race conditions.
In one of Yalantis’s projects, there was a case with a piece of a multithreaded program in Python. Then without changing the algorithm of the program, simply by rewriting a piece of code in Rust, the Yalantis team discovered a problem in the runtime that was unknown before. It was possible right at the compilation stage.
The essence of the problem was that the shared variable wasn’t fully synchronized between threads. This didn’t lead to an obvious error during testing, but in production, it could be detected indirectly by the logs and the work result. It was difficult to catch this bug as it has a non-deterministic nature.
Here is a Python code example where we are trying to do an operation with a common variable from two threads:
import time
import threading
shared_data = 10
def increment(increment_by):
global shared_data
local_counter = shared_data
local_counter += increment_by
time.sleep(1)
shared_data = local_counter
print(f'{threading.current_thread().name} increments x by {increment_by}, x: {shared_data}')
# creating threads
t1 = threading.Thread(target=increment, args=(5,))
t2 = threading.Thread(target=increment, args=(10,))
# starting the threads
t1.start()
t2.start()
# waiting for the threads to complete
t1.join()
t2.join()
print(f'The final value of x is {shared_data}')
*code snippet examples is taken from the guide to race condition
In the above example, the output The final value of x is will be either 20 or 15. Depending on which thread will be finished faster. This can be fixed by adding threading.Lock in the appropriate place, but when the code base is large, a developer can get confused and forget to add it.
Similar code in Rust already has corresponding Lock counterparts at the level of the type system, without which the program simply would not compile. The example below is written in Rust:
Thus, unlike with Python, in Rust code, it’s much easier to detect most bugs and avoid unexpected system failures in the production environment.
In short, Rust tools are robust, reliable, and easy to use. This can be very appealing to a Python developer who is interested in making the switch due to so many Rust benefits but doesn’t want to put in a ton of work starting from scratch again. If you have an existing Python code base that works “well enough”, you can significantly elevate the development by switching to Rust, at least partially.
