Matte surfaces would be harder to stick than gloss. Regular sign vinyl likes smooth, flat, non-porous surfaces. But it really should stick to anything, it'll just ruin the wallpaper or possibly rip the paint off the wall when removed. I'm worried about the quality of your vinyl overall. Does it stick to glass and metal? However, if your wall is dusty, the adhesive will collect all the dust and then simply fall back off. Make sure the walls are nice and clean, use distilled water on a rag. Normally I recommend isopropyl alcohol to de-grease the surface but that might take the paint off too. Certain types of outdoor wall paints are constantly shedding "powder" and will never be able to stick.
When you say it falls off fast, do you mean instantly, or in a few days? How fresh is the paint? It is not recommended at all to apply vinyl to a freshly painted surface, because even it it "feels" dry, it may be outgassing vapors long afterward. This will literally "push" the vinyl off the wall (or ruin the adhesive chemistry). You would have to ask the manufacturer when full cure is (if it's not listed on the can). Usually it's about a week for oil paints and a month for acrylic/latex/water based paints.