This paper presents a signal processing and deep learning framework for correlating physical and cardiac stresses using a single-lead ECG. Unlike previously proposed methods that rely on inertial sensors or fixed time-frequency transformations with handcrafted features to estimate physical workloads, our approach processes raw ECG segments directly through a learnable front-end that enables data-driven time-frequency representation learning. The model begins with a 1D convolutional layer capturing multi-scale temporal patterns inspired by the principles of continuous wavelet transforms and passes these representations to a ResNet backbone for end-to-end classification. The dataset includes ECG recordings from 54 subjects performing seven common daily activities: Sitting, Standing Up, Bending Over, Walking, Climbing Stairs, Jumping, and Running. These activities serve two classification objectives: recognition of the types of physical activity and classification into four predefined workload intensity levels, namely Low, Postural, Moderate, and Intense. The model is trained and evaluated using repeated stratified 5-fold cross-validation across multiple window sizes (2, 5, and 10 seconds). The experimental results demonstrate strong performance on both tasks, achieving a macro F1-score of approximately 99% for activity recognition and intensity classification under this evaluation setup.