After allowing Keldorn into your party, he will mention at some point that he has an estate in the Government District, and that if you happen to wander near it, he will point it out to you.

His estate is marked on the map, and once you head inside, Keldorn has a confrontation with his wife. In this dialog, Keldorn finds out that his wife has been seeing another man, named Sir William of Thorpe. Keldorn feels he is honor bound to confront Thorpe and wishes to travel to the Mithrest Inn, in the Waukeen's Promenade district.

Speaking to Thorpe had me confused for awhile. Keldorn was the leader of my party, but he refused to speak to him saying "I am waiting for another". I had to approach him with my main character to get him to speak. After a dialog, Keldorn comes to the conclusion that his neglect is to blame for all that has happened. Here you are given four choices.

Encourage Keldorn to fight, he will decline and state his desire to return to his estate. The net result is the same as the third choice, opting to go with him to his home. You can also release him in two different ways. One he will agree to meet you at the Radiant Heart guildhouse in three days time or you can simply release him for good.

If you chose to return with him to his home, and initiate dialogue with his wife, they will have a short conversation that leaves you with another three options. They all result in 15,500 experience points. Unless you want to be completely rid of him, choose either the option to give him a day, or keep him in your party. Either way, he promises to his wife to return to her after he helps you rescue Imoen, and she accepts.

If you chose to meet him at the guildhouse in three days time, you have the same three choices of releasing him, giving him a day, or keeping him in your party. However, this way did not result in any experience gain. Therefore, the best route seems to be to go to his home with him, and if you wish to keep him in your party, do so. You will not be penalized experience for keeping him in your party to help in the rescue of Imoen.