Steel–concrete–steel (SCS) structural systems have economic and structural advantages over traditional reinforced concrete; thus, they have been widely used. The performance of concrete made from recycled rubber aggregate from scrap tires has been evaluated since the early 1990s. The use of rubberized concrete in structural construction remains necessary because of its high impact resistance, increases ductility, and produces a lightweight concrete; therefore, it adds such important properties to SCS members. In this research, the use of different concrete core materials in SCS was examined. Twelve SCS specimens were subjected to push-out monotonic loading for inspecting their mechanical performance. One specimen was constructed from conventional normal weight concrete core, while the other specimens were constructed with modified core materials by either partial replacement of the coarse aggregate with crumb rubber (CR), the addition of oil palm fibre (OPF) to the concrete as a volume fraction of concrete, or both in the concrete cores. The investigated push-out specimens have a height of 450 mm and constructed from two hollow steel tubes with a square cross section of 100 mm and 5 mm in thickness which fixed to concrete prism using bolt end shear connectors. The detection of the mode of failure, load–slip as well as ductility behaviour, and the energy absorption capacity was investigated. The results revealed an improvement in the energy absorption (EA) capacity averagely by 55% for the specimen with 15% CR and 1.1% addition of OPF as a volume fraction of concrete in comparison with the reference specimens due to the high shear resistance.