These are some of the best answers I have seen on a forum. I agree with most of them but especially the idea that it is a chance for a new start, regardless of public or private. Have you considered the notion that children usually fit in where their parents fit in? Do you truly share these people's values or are you just frightened that your child won't get a good education if you don't send him private? Perhaps the children there are already in cliques arranged by their insecure parents or perhaps things just don't sit well for your son. There are obviously niggling doubts and your son could easily pick up that he doesn't really belong there (I am thinking about the most subtle 'vibrations' here - no I am not crazy but just couldn't find a better word).
Also consider that some children are going to be more intense and more sensitive, especially when they are surrounded by so many new personalities and experiences - they are going to be different to how they are at home. This doesn't mean that a new start would not work for him. Some of the same issues may arise, but you might work through them better in a different setting.
The thing I would do is to send him to the state school, but not put the responsibility for the change on his shoulders - it's too much of a burden. I would say something like, 'Guess what? The local school said they had room for more children' or 'Dad and I have decided that you will all (his siblings when they do) go to the local school because it is closer and I like the oval'. This way, he will think it is just a great lucky thing that has happened rather than something he has made happen. This lifts the burden from him and also helps if it isn't the super place he thought it would be - because no place is all wonderful.
The deposit and books you talk about are a drop in the ocean that is going to be the life of your three children. Think of the fees you will save over the years. Take them skiiing in Japan instead when they are older. Pull back a bit from the pressure, put your child in the local school and hopefully watch him thrive.