High walls gave those inside a castle a tremendous advantage.

At first castle walls were wooden, making them cheap and quick to build, but they were vulnerable to arson. Stone walls followed, and with each generation they grew thicker and taller. From the mid-13th century, many castles had concentric rings of walls, one encircling the other. Caerphilly Castle is the earliest example of this in Britain, and the largest castle ever built in Wales. Low outer walls served as barriers against siege towers and battering rams. If attackers managed to break through this outer ring of defense, castle defenders could retreat behind high inner walls.

Corner towers stood out from the walls, giving defenders a better perspective on enemy movements. Windows were rare; instead, slits called loopholes were built for archers. Sometimes builders thickened walls low to the ground to protect them from battering rams. Often, these walls sloped away at the base to redirect objects dropped from the top of the castle wall, ricocheting them out at soldiers on the ground.

Because they had walls to protect them, castle defenders would sometimes hunker down and try to wait out their attackers. Those inside made sure they could be self-sufficient when cut off from the outside world by a siege. They built wells and kept livestock inside their walls, guaranteeing fresh water and fresh meat during a siege. They also salted foods such as bacon and fish and stored grains and beans by the barrel-full.

Castle garrisons also stockpiled weapons, for reinforcements often could not get through. If a siege continued into winter, castle dwellers had more protection from the elements than attackers, and if their rations held up, also more food to sustain them.

Those besieged inside a castle often negotiated time frames for surrender with the enemy. For example, a castle garrison might tell an attacking army that they would surrender if reinforcements did not appear by a specified date. This would save lives on both sides and avoid the steep financial costs of a siege as well.