[verb (ば-form)] + よかった [verb ない-form 〜ない → 〜なければ] + よかった
ばよかった is regret: "I should have…", "I wish I had…". It is the ば-form of a verb followed by よかった, the past of いい – word for word, "it would have been good if…". The thing you regret is something you didn't do.
For something you did and wish you hadn't, put the verb in the negative: ない becomes なければ.
The past is carried by よかった, so the verb before it stays in the ば-form. For する that is すれば.
These go beyond your own regret.
With のに, the regret turns outwards. Said to someone else, it is a mild reproach: you should have, so why didn't you?
Asked as a question, it looks back for what would have worked.