Cython - Export my lib to a .so file and use in a C program - Symbols not found

ONLINECAST

2 min read

·

29-08-2024

25

0

Share

Cython: Exporting a Library and Using it in C

This article explores the process of creating a Cython library (.so) and using it in a C program. We'll address a common issue: exported functions not being visible to the C application, and provide a solution using an example.

Problem: You've created a Cython library, compiled it to a .so file, and are trying to use it in a C program. However, the C program cannot find the Cython functions you've exported.

The Root Cause: The issue often lies in the way Cython handles function naming and the lack of explicit export declarations.

Example:

Let's assume your Cython file (process_csv_file.pyx) looks like this:

import cython

@cython.cfunc
def processCsvFile(csv: cython.p_char, selectedColumns:cython.p_char, rowFilterDefinitions:cython.p_char):
    # ... Your CSV processing logic ...
    return 

@cython.cfunc
def processCsv(csv_data:cython.p_char, selectedColumns:cython.p_char, rowFilterDefinitions:cython.p_char):
    # ... More CSV processing ...
    return 

The Solution: Explicit Exports and Naming Conventions:

1. Explicit Exports:

   • Use the cdef extern from construct to explicitly define your functions as C functions.
   • In your process_csv_file.pyx, add the following:

   cdef extern from "process_csv_file.h":
       void processCsvFile(char* csv, char* selectedColumns, char* rowFilterDefinitions)
       void processCsv(char* csv_data, char* selectedColumns, char* rowFilterDefinitions)
   

   • Create a header file process_csv_file.h with the following content:

   void processCsvFile(char* csv, char* selectedColumns, char* rowFilterDefinitions);
   void processCsv(char* csv_data, char* selectedColumns, char* rowFilterDefinitions);
   

2. Naming Conventions:

   • Cython adds underscores to generated function names. To avoid this, use the __name__ attribute.
   • Modify your Cython file:

   import cython
   
   @cython.cfunc
   def processCsvFile(csv: cython.p_char, selectedColumns:cython.p_char, rowFilterDefinitions:cython.p_char):
       # ... Your CSV processing logic ...
       return
   
   processCsvFile.__name__ = "processCsvFile"
   
   @cython.cfunc
   def processCsv(csv_data:cython.p_char, selectedColumns:cython.p_char, rowFilterDefinitions:cython.p_char):
       # ... More CSV processing ...
       return
   
   processCsv.__name__ = "processCsv"
   

3. Modified Build Process:

   • Update your build command to include the header file:

   cython --embed csv_reader_lib/process_csv_file.py -o libcsv.c
   gcc -o libcsv.so -shared -fPIC libcsv.c process_csv_file.h $(python3-config --cflags --ldflags) -lpthread -lm -lutil -ldl -lpython3.12
   cp libcsv.so /usr/local/lib/
   

4. C Program Integration:

   • Include the header file in your C program and call the functions:

   #include "process_csv_file.h"
   
   int main() {
       // ...
       processCsvFile("path_to_csv", "selected_columns", "row_filter_definitions");
       // ...
       return 0;
   }
   

Additional Considerations:

• Compile Options: The -fPIC flag ensures position-independent code, which is essential for shared libraries.
• Library Path: Ensure the library path (/usr/local/lib in this case) is in your system's library search path.
• Dynamic Linking: You might need to use dlopen and dlsym to dynamically load and access functions from your Cython library if you are not linking the library statically.

Key Points:

• Explicit Exports: Always explicitly define functions using cdef extern from in your Cython code.
• Naming Conventions: Control function names by using __name__ to avoid Cython's default naming scheme.
• Header File: Create a header file to define the exported functions for both Cython and your C code.

By following these guidelines, you can successfully create a Cython library and seamlessly integrate it into your C programs.