Regarding the buildup on your ironing surface. I have a muslin piece that is just for starching. I lay it over the ironing surface before doing any starching. BTW: I'm a starcher (is that a word?) from way back when my Grandma taught me how to quilt with little pieces. She was way ahead of her time. :)

I always spray the back side of fabrics, to me starch is just as important tool as my sewing machine.
The last can I got says on the front no residue and they are telling the truth.
Those white specks are just cornstarch and will wash off or wear off.

I use spray sizing rather than starch. You can get it at your grocery store.

I have never used starch.

I use either Niagara non-aerosol spray starch or Best Press. The hard part is being patient and not ironing immediately after spraying. You need to let the starch penetrate the fabric. It really does help in getting flat seams and less distortion on the fabric, especially on bias edges. I too use a piece of muslin on my pressing surface to keep it clean. It's easy to throw that piece of muslin in the washer every now and then. If my iron gets gunky on the bottom I use Mr. Clean Magic Eraser on a cold iron. It works great at getting the sole plate shiny clean again. I have a shiny metal sole plate on my iron and not a Teflon coated one so I don't know how it would work on the coated sole plates.

Like others have already said, you didn't wait long enough before you ironed. Besides that, the white flakes just pretty much brush off & will come out in the wash.

You can buy spray starch that is non flaking.

I use Faultless MAXXm great heavier starch when you need it. I also use Best Press, it is lighter. I have never had while flaking with either one. With other flaking starches I used to iron from the back side...still no flakes.

Wish this had posted a few days ago, as I just purchased Best Press. Now I find out that regular "store bought" starch works better. Live and Learn.....

You can make your own. I use Sta-Flo (very cheap at the grocery store) and dilute it one Sta-Flo and 4 parts water. Works like a charm and soaks in quite quickly if you want to spray and iron. A little spreading with the fingers helps, too.
So the other day I made a new bottle of this starch, started using it, and found I was getting flakes. I looked at my bottle and saw that all the starch was at the bottom. It had not been mixed. The spray tube was picking up only the starch. I shook it up and it works great. Now I try to remember to shake it each time I use it, but it doesn't really separate once it is well mixed. This product works great. Sometimes I soak a piece of fabric in it, wait for it to dry a little or put it in the dryer, and iron. Makes beautiful crisp fabric to work with.