The outside pieces will be really large compared to the inside pieces. May not give you the crazy quilt look you want.
We did this with one of our first quilts, but sewing 4 triangles to the sides of a square to make a bigger square, repeat, etc. Came out looking really weird, but someone liked it enough to buy it! And in light green and black at that!