I've seen brick by brick construction before in peruvian shantytowns, but they lived in the houses the entire time.

They would start with bamboo stakes, woven bamboo mats and sheets of black PVC plastic to construct temporary walls and and claim squatting rights on the land. Then they would slowly buy bricks and stack them up to supplement the walls.

Eventually they would buy cement to construct proper walls and proper roofing materials (the black PVC sheeting was used as a roof until then)

In the long term, they might repeat the process and start on a second story.