ABBOTTABAD, Pakistan (AP) ― When a woman involved in a polio vaccine drive turned up at Osama bin Laden’s hideaway, she remarked to the men behind the high walls about the expensive SUVs parked inside. The men took the vaccine, apparently to administer to the 23 children at the compound, and told her to go away.

The terror chief and his family kept well hidden behind thick walls in this northwestern hill town they shared with thousands of Pakistani soldiers. But glimpses of their life are emerging ― along with deep skepticism that authorities didn’t know they were there.

Although the house is large, it was unclear how three dozen people could have lived there with any degree of comfort.

Neighbors said they knew little about those inside in the compound but bin Laden apparently depended on two men who would routinely emerge to run errands or to a neighborhood gathering, such as a funeral. There were conflicting details about the men’s identities. Several people said they were known as Tariq and Arshad Khan and had identified themselves as cousins from elsewhere in northwestern Pakistan. Others gave different names and believed they were brothers.

Arshad was the oldest, and both spoke multiple languages, including Pashto and Urdu, which are common here, residents said.

As Navy SEALs swept through the compound early Monday, they handcuffed those they encountered with plastic zip ties and pressed on in pursuit of bin Laden. After killing the terror leader, his son and two others, they doubled back to move nine women and 23 children away from the compound, according to U.S. officials.

Those survivors of the raid are now “in safe hands and being looked after in accordance to the law,” the Pakistani government said in a statement. “As per policy, they will be handed over to their countries of origin.” It did not elaborate.

Also unclear was why bin Laden chose Abbottabad, though at least two other top al-Qaida leaders have sheltered in this town. The bustling streets are dotted with buildings left over from British colonial days. These days it attracts some tourists, but is known mostly as a garrison town wealthier than many others in Pakistan.

Bin Laden found it safe enough to stay for up to six years, according to U.S. officials, a stunning length of time to remain in one place right under the noses of a U.S.-funded army that had ostensibly been trying to track him down. Most intelligence assessments believed him to be along the Afghan-Pakistan border, perhaps in a cave.

Construction of the three-story house began about seven years ago, locals said. People initially were curious about the heavily fortified compound ― which had walls as high as 18 feet topped with barbed wire ― but over time they just grew to believe the family inside was deeply religious and conservative.

The Pakistani government also pushed back at suggestions that security forces were sheltering bin Laden or failed to spot suspicious signs.

“It needs to be appreciated that many houses (in the northwest) have high boundary walls, in line with their culture of privacy and security,” the government said. “Houses with such layout and structural details are not a rarity.”

The house has been described as a mansion, even a luxury one, but from the outside it is nothing special. Bin Laden may well have been able to take in a view of the hills from secluded spots in the garden, though.

The walls are stained with mold, trees are in the garden and the windows are hidden. U.S. officials said the house had no Internet or phone connection to reduce the risk of electronic surveillance. They also said residents burned their trash to avoid collection.

Those who live nearby said the people in bin Laden’s compound rarely strayed outside. Most were unaware that foreigners ― bin Laden and his family are Arabs ― were living there.

Khurshid Bibi, in her 70s, said one man living in the compound had given her a lift to the market in the rain. She said her grandchildren played with the kids in the house and that the adults there gave them rabbits as a gift.

But the occupants also attracted criticism.

“People were skeptical in this neighborhood about this place and these guys. They used to gossip, say they were smugglers or drug dealers. People would complain that even with such a big house they didn’t invite the poor or distribute charity,” said Mashood Khan, a 45-year-old farmer.

Questions persisted about how authorities could not have known who was living in the compound, especially since it was close to a prestigious military academy.

As in other Pakistani towns, hotels in Abbottabad are supposed to report the presence of foreigners to the police, as are estate agents. Abbottabad police chief Mohammed Naeem said the police followed the procedures but “human error cannot be avoided.”

Reporters were allowed to get as far as the walls of the compound for the first time, but the doors were sealed

By early Wednesday, roads leading to the compound had once again been sealed off, this time with even tighter security.

Neighbors earlier showed off small parts of what appeared to be a U.S. helicopter that malfunctioned and was disabled by the American strike team as it retreated. A small servant’s room outside the perimeter showed signs of violent entry and a brisk search. Clothes and bedding had been tossed aside. A wall clock was on the floor, the time stuck at 2:20.

Abbottabad has so far been spared the terrorist bombings that have scarred much of Pakistan over the last four years.

Like many Pakistani towns where the army has a strong presence, Abbottabad is well-manicured, and has solid infrastructure. Street signs tell residents to “Love Pakistan.” The city also is known for its good schools, including some that were originally established by Christian missionaries.

Little girls wear veils while carrying Hannah Montana backpacks to school. Many houses in the outlying areas have modern amenities, but lie along streets covered with trash. Shepherds herd their flock of sheep along dusty roads just a few hundred yards from modern banks.

Al-Qaida’s No. 3, Abu Faraj al-Libi, lived in the town before his arrest in 2005 elsewhere in northwest Pakistan, according to U.S. and Pakistani officials. Earlier this year, Indonesian terror suspect Umar Patek was nabbed at a house in the town following the arrest of an al-Qaida courier who worked at the post office. It is not clear whether Patek had any links with bin Laden.

Western officials have long regarded Pakistani security forces with suspicion, chiefly over their links to militants fighting in Afghanistan. Last year, U.S. Secretary of State Hillary Rodham Clinton caused anger in Pakistan when she said she found it “hard to believe” that no one in Islamabad knows where the al-Qaida leaders are hiding and couldn’t get them “if they really wanted to.”

But al-Qaida has been responsible for scores of bloody attacks inside Pakistan, including on its army and civilian leaders. Critics of Pakistan have speculated that a possible motivation for Pakistan to have kept bin Laden on the run ― rather than arresting or killing him ― would be to ensure a constant flow of U.S. aid and weapons into the country.

Suspicions were also aired in Pakistani media and on the street Tuesday.

“That house was obviously a suspicious one,” said Jahangir Khan, who was buying a newspaper in Abbottabad. “Either it was a complete failure of our intelligence agencies or they were involved in this affair.”