This paper presents a promising solution for scalable hardware architecture for implementing AI engines in wideband radio systems with polyphase systolic arrays on a Stratix-10 AX system on chip (SoC) FPGA. The architecture is designed using matrix-vector multiplier (MVM) cores from the scratch to accommodate the configurability, scalability, and the portability for any FPGA or ASIC platforms. A 16 × 8 MVM core supporting 16 digital datapaths in parallel has been implemented and verified on Stratix-10 AX, achieving an effective 4 GHz instantaneous processing bandwidth. The significance of the architecture is that each atomic block performs fully parallel multiply-accumulate (MAC) operations, and multiple blocks can be instantiated to form larger systolic arrays. This enables linear scaling of performance up to terascale MAC-throughput levels required by next-generation spectrum-sensing and edge-AI accelerators for wideband radio. Furthermore, the integration of polyphase DSP structures with dense-layer inference in a single hardware provides a low-latency and energy-efficient platform for emerging 6G and beyond applications.