Radiologists typically have a hard time to classify the breast cancer, which leads to unnecessary biopsies to remove suspicions, and this ends up in adding exorbitant expenses to an already burdened patient and health care system. As well as early detection and diagnosis can save the lives of cancer patients. In this paper, a computer-aided diagnosis (CAD) system based on hybrid intelligence framework using Gabor wavelet-based deep learning convolutional neural network (GW-DL-CNN) for the detection and classification of breast cancer in mammographic images is proposed. In addition, a machine learning framework with Gabor wavelet-based support vector machine (GW-SVM) also implemented. Both, GW-SVM and GW-DL-CNN models are proposed to help the radiologist in a much better way to detect and classify the breast cancer from mammographic images. Further, Chan-Vese (C-V) features-based level set segmentation also utilized for segmenting the objects without clearly defined boundaries in mammographic images. The unified features extracted from C-V and GW are fed into an architecture of DL-CNN to classify the type of breast cancer such as malignant, benign, or normal using fully complex valued relaxation network (FCRN) classifier. The proposed frameworks of GW-SVM, GW-DL-CNN with FCRN classifier is achieved the model accuracy of 98.6%, specificity of 98%, sensitivity of 98% and F1-Score is 97.08% respectively.