Hi there! There are loads of free activities in Wallingford. Bearing in mind the ages of your two I would recommend the toddler and baby group at the Baptist church on Thames Street on a Tuesday morning from 10am and there are two groups on a Wednesday - one at St Mary's from 10am and one at the Methodist Church from 9.30am. Mondays is a group run by the children's centre at the Masonic hall, plus there are a whole load more groups just for babies that might not be suitable because of having a toddler with you. Hope that helps - let me know if you need more info!
The Methodist Church is actually a quid for one child and 50p per additional child to attend, but you get a hot drink and grown up biscuits and a cold drink and fruit and biscuits for small people, so it's good value.