The fascinating thing is that speaking in terms of expectation and cliché, this isn’t what you think Peter will say; he’s the “good one” in this family, after all, and should encourage Nathan to do his duty as an ADA, put the greater good ahead of personal feelings, go after the mobster.

I think I have a slightly different take on the conversation, actually.

So, clearly, Nathan does not want to take down his father, it's clear from the conversation and confirmed by later evidence that in spite of the fact that he recognizes that Pop's relationship with Linderman was a bad, bad thing, he still vastly respected and loved his father. I think what he's doing in that conversation is basically asking Peter's permission to not go after Linderman. He knows he should, but he can't stand it, and Peter is, ultimately, his conscience. So he's looking for an out.

And Peter, though he, in actuality, would be thrilled for someone to take down Linderman, sees how bad it would hurt Nathan, so he lets him off the hook.

Also, I think Nathan's line about Peter saying defending criminals is the same as being one is a hugely important character hint. I think it's very important that Nathan is an ADA rather than a defense attorny, because I think, unlike most other things Nathan's done, that's probably a pretty large break with the family tradition right there. He didn't go so far as to become a nurse, granted, but he refused to go where the big money was, and where the family hertitage was, and instead, went for a job in civil service. And it seems to me, that's probably precisely because of Peter--his conscience's--stance on defense attornies. And of course, that idea is echoed in .07% when Nathan says he doesn't know who he is without Peter.

Peter is his hero, omg, it's so cute!!1111!!eleventy! < / fangirl moment > *composes self* Ahem. Yeah.