Fabric is only one kind of material. There is also wood, brick, insulation. Those are all material (for building-- can you tell I am married to a former contractor?) There is also landscaping material, roofing material, get it? Material is what you use to do a job, but it's not always fabric. Fabric, however, always means fabric. I never ask for material, as it's too general a term. If I mean fabric, I say fabric.