05. Hausa Language Models

Hausa presents dual-script challenges that complicate NLP deployment. While formal education and official contexts use Latin script (boko), traditional and religious communities predominantly use Arabic script (ajami). Production systems must handle both scripts and the code-switching patterns that emerge in multilingual communities.

Available resources include the HausaNLP project's models, Bible corpus translations, and BBC Hausa news data. The Hausa Voice corpus provides speech recognition training data. However, resource distribution remains uneven—Arabic script processing capabilities lag significantly behind Latin script, creating deployment gaps for users who prefer or exclusively use ajami.

Script conversion between Latin and Arabic representations requires careful handling. Character mapping tables must address multiple transliteration conventions, as different communities use varying spellings for the same sounds. Contextual ambiguity exists—some characters appear similar in both scripts but represent different phonemes. The conversion pipeline must expose uncertainty to users rather than silently selecting an incorrect rendering.

# Hausa Latin-Arabic script converter with confidence scoring
import re

class HausaScriptConverter:
    # Mapping tables for Latin to Ajami conversion
    # Multiple mappings exist for some sounds
    LATIN_TO_AJAMI = {
        'a': 'ا', 'b': 'ب', 'd': 'د', 'e': 'ۋ',
        'f': 'ف', 'g': 'گ', 'h': 'ح', 'i': 'ي',
        'j': 'ج', 'k': 'ك', 'l': 'ل', 'm': 'م',
        'n': 'ن', 'o': 'ۇ', 'r': 'ر', 's': 'س',
        'sh': 'ش', 't': 'ت', 'ts': 'تس', 'u': 'ۋ',
        'w': 'و', 'y': 'ي', 'z': 'ز', ' \' ': 'ع',
    }
    
    def __init__(self):
        self.unmatched = []
    
    def latin_to_ajami(self, text: str, include_diacritics: bool = False) -> dict:
        """Convert Hausa Latin script to Ajami with confidence scoring."""
        result = []
        confidence_scores = []
        self.unmatched = []
        
        i = 0
        while i < len(text):
            # Check for digraphs first
            if i < len(text) - 1:
                digraph = text[i:i+2].lower()
                if digraph in self.LATIN_TO_AJAMI:
                    result.append(self.LATIN_TO_AJAMI[digraph])
                    confidence_scores.append(0.95)
                    i += 2
                    continue
                elif text[i].lower() in self.LATIN_TO_AJAMI:
                    char = text[i].lower()
                    result.append(self.LATIN_TO_AJAMI[char])
                    confidence_scores.append(0.98)
                    i += 1
                else:
                    # Keep original character
                    result.append(text[i])
                    confidence_scores.append(0.0)
                    self.unmatched.append((i, text[i]))
                    i += 1
            else:
                if text[i].lower() in self.LATIN_TO_AJAMI:
                    result.append(self.LATIN_TO_AJAMI[text[i].lower()])
                    confidence_scores.append(0.98)
                else:
                    result.append(text[i])
                    confidence_scores.append(0.0)
                    self.unmatched.append((i, text[i]))
                i += 1
        
        avg_confidence = sum(confidence_scores) / len(confidence_scores) if confidence_scores else 0
        
        return {
            'text': ''.join(result),
            'confidence': avg_confidence,
            'unmatched_chars': self.unmatched,
            'warnings': self._generate_warnings()
        }
    
    def _generate_warnings(self) -> list[str]:
        warnings = []
        if len(self.unmatched) > 5:
            warnings.append("High number of unmatched characters - verify input script")
        non_latin_ratio = len(self.unmatched) / max(1, sum(1 for c in self.unmatched if ord(c[1]) > 127))
        if non_latin_ratio > 0.3:
            warnings.append("Possible mixed script or formatting characters detected")
        return warnings

# Example usage
converter = HausaScriptConverter()
result = converter.latin_to_ajami("Sannu da zan baka")
print(f"Conversion: {result['text']}")
print(f"Confidence: {result['confidence']:.2f}")

Hausa NLP deployments often incorporate Islamic religious content due to corpus availability. This creates biased models that perform well on religious queries but poorly on agricultural, health, or commercial topics. Careful evaluation using domain-specific test sets reveals these biases. Countermeasures include data mixing strategies that oversample underrepresented domains and model fine-tuning on curated agricultural or health corpora.

Northern Nigeria's infrastructure constraints intensify deployment challenges. Grid power availability drops to 8-12 hours daily in some areas. Mobile data costs remain high relative to average income. Device hardware lags behind global averages, with significant populations using MediaTek-based phones with limited RAM. These constraints make model compression not merely beneficial but mandatory for practical deployment.