For the most part yes, though there's often difficulty saving without a specific type of memory card. This thread from the Gamespot forums goes into a lot more detail: Can a PS2 play all PS1 games? - Games Discussion - GameSpot. read more
On the newer, slimline PStwo's, this ps1 chip and its RAM are removed, and a software emulator is used for PS1 games. Further, the EE/GS are overclocked to compensate for the inefficiencies of the emulator. So this BC is imperfect, and could also cause problems for those ps2 games that used that chip as well. read more