> Not if there are incentives for the algorithms to be subject to public scrutiny.
The problem is that we don't have that transparency now, and aren't particularly likely to in the future. [1] Particularly for software-as-a-service: I can't audit Facebook's algorithms, despite their effects on what my friends and family see and discuss. And I can't easily audit the black box models used to make public policy, despite the value of transparency there, either.
Also, it's far from clear that transparency into an AI's algorithms buys you very much. A mathematically inclined college freshman has no trouble understanding how to train a deep neural network today, but it's at the cutting edge of research to understand what function a trained network is actually computing.
Scale that up in complexity a few orders of magnitude, and the thing may as well be a human. That is, you'll likely have broad understanding of its "motives", but we have that for humans today and it doesn't help that much. I understand Facebook wants to sell advertising, but that transparency isn't sufficient for understanding why a particular message is delivered to your followers while another isn't.
The problem is that we don't have that transparency now, and aren't particularly likely to in the future. [1] Particularly for software-as-a-service: I can't audit Facebook's algorithms, despite their effects on what my friends and family see and discuss. And I can't easily audit the black box models used to make public policy, despite the value of transparency there, either.
[1] http://www.theatlantic.com/technology/archive/2015/01/the-ca...