Build Cython Extensions

This skill provides guidance for building Cython extensions and resolving compatibility issues, with particular focus on numpy version compatibility problems.

When to Use This Skill

• Building or compiling Cython extensions (.pyx files)
• Fixing numpy compatibility issues in Cython code
• Migrating Cython projects to work with numpy 2.0+
• Resolving deprecated numpy type errors (np.int, np.float, np.bool, etc.)
• Troubleshooting Cython compilation failures

Key File Types to Examine

When working with Cython projects, always examine ALL relevant file types:

| Extension | Description | Must Check | |-----------|-------------|------------| | .pyx | Cython implementation files | Critical - Often contain numpy calls | | .pxd | Cython declaration files | Yes - May contain type declarations | | .py | Python files | Yes - May use deprecated types | | setup.py | Build configuration | Yes - Defines compilation settings | | .c / .cpp | Generated C/C++ files | Only if debugging compilation |

Critical Pitfall: Never limit searches to only .py files when fixing numpy compatibility. The .pyx files are Cython source code and frequently contain the same deprecated numpy type references.

Approach for Numpy 2.0+ Compatibility

Deprecated Types to Replace

| Deprecated | Replacement | |------------|-------------| | np.int | np.int_ or int | | np.float | np.float64 or float | | np.bool | np.bool_ or bool | | np.complex | np.complex128 or complex | | np.object | np.object_ or object | | np.str | np.str_ or str |

Search Strategy

1. Search without file type restrictions to capture all occurrences:

   Grep for patterns like "np\.int[^0-9_]" across all files
   

2. Explicitly search Cython files:

   Search specifically in *.pyx and *.pxd files
   

3. Check import statements in .pyx files - they often import numpy and use deprecated types

Fix and Recompile Workflow

1. Identify all .pyx files in the project
2. Search each file for deprecated numpy types
3. Apply fixes to ALL files (both .py and .pyx)
4. Recompile the Cython extensions after making changes to .pyx files
5. Run verification tests

Verification Strategy

Import Testing Is Insufficient

Simply testing that a compiled module imports successfully does not verify the code works correctly. A module can import but fail when its functions are called.

Recommended Verification Steps

1. Identify all Cython modules in the project
2. For each module:
   • Verify import succeeds
   • Call at least one core function from each module
   • Pass actual data to exercise numpy operations
3. Run the project's test suite if available
4. Create a verification script that exercises key functionality:

   # Example verification pattern
   import numpy as np
   from module import cython_function
   
   # Test with actual numpy arrays
   test_data = np.array([1, 2, 3], dtype=np.int64)
   result = cython_function(test_data)
   assert result is not None
   

Test Coverage Awareness

• Repository tests may not cover all Cython code paths
• Passing tests does not guarantee all Cython functionality works
• Explicitly test functions that use numpy types

Common Pitfalls

1. Narrow Search Scope: Using file type filters (e.g., type: "py") that exclude .pyx files
2. Premature Success Declaration: Assuming success after imports work or basic tests pass
3. Missing Recompilation: Forgetting to recompile after fixing .pyx files
4. Incomplete Pattern Matching: Missing variations like numpy.int vs np.int
5. Ignoring Warning Signs: If compilation succeeds "surprisingly" easily, verify the compiled code actually runs correctly

Systematic Workflow

1. Discovery Phase

   • List all .pyx, .pxd, and .py files
   • Identify the build system (setup.py, pyproject.toml, etc.)
   • Check numpy version requirements

2. Analysis Phase

   • Search ALL source files for deprecated patterns
   • Document every occurrence before fixing
   • Note which files need recompilation

3. Fix Phase

   • Apply fixes to all identified locations
   • Ensure consistency in replacement types
   • Update any type annotations or docstrings

4. Build Phase

   • Clean previous build artifacts
   • Recompile all Cython extensions
   • Watch for compilation warnings

5. Verification Phase

   • Test each Cython module individually
   • Run the full test suite
   • Execute functions with real numpy data
   • Verify no runtime AttributeError for numpy types