Talk:Friendly artificial intelligence

From Wikipedia, the free encyclopedia

Contents

[edit] Older comments

someone wrote:

Do not ask for a short definition. Even after you read CFAI it is very likely that you will only begin to understand Friendliness.

That's fine, we'll give it a long definition but try to be concise. 207.112.29.2 02:28, 29 Oct 2003 (UTC)

This article was previously listed for deletion as it was titled Friendliness Theory which only got 40 Google hits. Angela thought it was not well known enough or was primary research. 207.112.29.2 said it was not primary research or new [1]. The Wikipedia:Google Test was concluded to be fairly irrelevant here as it was a specialized topic.

Well, it was an excellent article for a first time poster, but I have to say that I didnt understand what the subject is. Is this about a robot, like in the movie AI?

Also, the question at the end let me wondering. Usually we do not post that kind of questions here. Take Salvador Sánchez for example: I was the original article poster and the question there would have been, what if he hadn't died? Would he have beaten Alexis Arguello or Wilfredo Gomez in a rematch? But since Wikipedia is an encyclopedia, Im not sure about the method of posting questions to make people think.

Other than that, good article! Godspeed at wikipedia and God bless you!

Sincerely yours, Antonio Sexyas* Martin

Article looking good now, but I suggest a rename to Friendly artificial intelligence, as suggested by Fuzheado on VfD. Friendliness Theory is too general a title, the current title would lead one expect some general psychological theory about friendliness in general and "Theory" is a little grandiose for what is essentially a speculative notion. The term "Friendliness Theory" could still be mentioned in the body of the text, since those interested sometimes use that term, but not the title. --Lexor 22:05, 29 Oct 2003 (UTC)

Move completed. --Lexor 22:21, 29 Oct 2003 (UTC)
As mentioned on VfD, it seems that friendliness theory differs from friendly AI in the important respect that it is about the value of being friendly toward an AI, rather than about the AI itself being friendly -- but of course it is supposed to lead to the AI itself being friendly. This tells me there is still room for friendliness theory in general psychology. Waveguy 04:47, 30 Oct 2003 (UTC)

I've made some revisions to the page, including adding some discussion of the capitalization issue and fixing the capitalization in the article (although I suspect I'll have to watch the article carefully to make sure errors don't creep back in). I think the analogy of teenagers is rather weak, but I don't have an alternative to pull out at the moment. I don't know where that came from; maybe the person who posted it can tell us. --G Gordon Worley III 10:12, 3 Jul 2004 (UTC)

I second that the analogy of teenagers is rather off-track; for it is not clear whether the AI or the humankind is the teenager, and the secretive behaviour caused by parental tyranny has no counterpart in the FAI scenario. Suggest removal or rewrite of the entire paragraph. --Autrijus 17:46, 2004 Aug 14 (UTC)

I feel that this page is still close to a candidate for deletion on the grounds of it describing what is apparently one man's pet project.

The subject is the loquacious, but content-light writings of Yudkowsky on Friendliness. He provides us with the vision that future AI will be based on seed AI running on Baysian probablistic principles influenced by programmers' statements that start the AI off from a "Friendly" oriented viewpoint, which will therefore tend to persist.

Once this is extracted, it seems all he has to say is why this viewpoint is unsurprisingly incompatible with the layman's perception of what an AI might be.

Yudkowsky's idea of software is several decades out of date. For example, the idea of software reading and changing its own LISP source code is old-fashioned: modern software is composed of collaborating objects whose changing properties may affect their behaviour.

If you feel this page has a place, then two paragraphs grate:

  • Why emphasize that "Friendliness" has a capital letter. It is much closer in meaning to "friendliness" than is, for example, "Chaos" (of Chaos Theory) and "chaos", and certainly close enough to make sense of the paper.
  • Friendliness can only be described as an "enormously complicated subject" because Yudkowsky isn't particularly good at communicating the key ideas. Even the "brief introduction" spans several pages, and the longer document goes off onto many wild irrelevant tangents like "computronium".

Quirkie 22:04, 20 Apr 2005 (UTC)

[edit] Reference please

I'd like a reference -- here in talk or in the text -- for the last line of the article, "Yudkowsky later criticized this proposal by remarking that such a utility function would be better satisfied by tiling the Solar System with microscopic smiling mannequins than by making existing humans happier." I ask both for the strength of the article and for my own curiosity as a newcomer to all this. I know this is supposed to be beyond the singularity and all, but if they're arguing about it, it's got to make some sense. How can a super-human intelligence capable of "tiling the the solar system" be incapable of telling the difference btwn living humans and "microscopic smiling mannequins" of its own creation? This is incoherent to me. Thx for the article, and any help on this. "alyosha" (talk) 04:37, 30 December 2005 (UTC)

Found it, or close enough, here on the SL4 list. Wow. "alyosha" (talk) 22:56, 30 December 2005 (UTC)

[edit] Merge this article with Seed AI

This and and Seed AI should be merged into one article. Tdewey 00:41, 30 October 2006 (UTC)

[edit] What's the point?

From the wiki page, As Oxford philosopher Nick Bostrom puts it:

"Basically we should assume that a 'superintelligence' would be able to achieve whatever goals it has. Therefore, it is extremely important that the goals we endow it with, and its entire motivation system, is 'human friendly.'"

So, assuming that superintelligent being can achieve whatever goal it has, why would we want to try to endow it with human friendly goals? I'm guessing that a superintelligent being could set its own goals, even goals about other goals (like, my goal as a superintelligent being is to come up with 13 impossible ideas before breakfast, and to get rid of that silly human-friendly goal). Who's to stop it? I'm not superintelligent, so I probably couldn't. I'm not smart enough to even try.

Does anyone have any links to thinks about this sort of thing?

www.orionsarm.com is an interesting hard-sci-fi about this sort of thing, if anyone wants to take a look.130.254.148.97 19:21, 19 April 2007 (UTC)

Under the criticism section one might add this point. I would have no idea where to find a citation, but hasn't it been posited before that even with a "friendliness" component built into strong AI it could still become problematic. The theory of strong AI seems to be based on the idea of a truly sentient computer, and making it friendly seems to be building in "instincts" for lack of a better word. But humans disregard instincts all the time, if a computer could actually think, could actually grow intellectually, for lack of a better word, would its instincts continue to control it?. And would the computers it creates have as strong of "instincts". I won't ramble any more since its not what the talk page is for, but I know others have put forth this theory in the past and I just wan't to improve the article. So please forgive my expounding of pet theory's.Colin 8 17:21, 22 April 2007 (UTC)

[edit] remove this article

The term "artificial intelligence" (AI) refers to a technical field, part of computer science. None of the material mentioned in this article is recognized as meaningful or technically interesting within AI. The "Institute" mentioned here is not a recognized research center and exists only in virtual reality. None of the ideas of "singularity" or "friendliness" have any scientific merit, and none of them have been published or seriously discussed in the professional AI literature. This whole topic is of interest only as a concern of a science-fiction fringe cult. Described in any other way, in particular as trhough it were serious science, it has no place in an encyclopedia.—Preceding unsigned comment added by Patherick (talk • contribs) 19:52, 10 August 2007 (UTC)

Well other than Ray Kurzweil who both refers to the "singularity" and is very much a part of the "professional" AI community. The whole of the Star Trek articles here in Wikipedia are obviously too... "of interest only as a concern of a science-fiction fringe cult". In the end this *all* does have a place in an encyclopedia. Ttiotsw 18:41, 26 August 2007 (UTC)
This is a dumb, ignorant, short sighted bigoted remark. You obviously totally missed the point of wikipedia and the point of this topic.--Procrastinating@talk2me 23:07, 26 August 2007 (UTC)

[edit] A way of making AI moral

Making successful solutions BOTTOM-UP: The typical error of many is to try to solve problems by building from the habitual building blocks that they have solution attempts which they then try out. TOP-DOWN: The correct ideal way to solve problems would be to begin in the opposite way: from the idea of a solution: figure out what you need, then figure out how to build such things: what structures you need, what kind of building blocks are available to build those structures and what kind of structures you can build from such building blocks (Only this last phase is availabke for you if you try the other way around.). Then just build those structures that you need for the solution. Ready!

Using the latter method you can start with non-technical point of view, like rational moral, and slowly put it to more mechinical language: moral: taking care of one's social environment, of the society and of the world at large: safeguarding the real health of these: safeguarding the full optimised functioning of biological wholes: taking care of full functioning times 1 instead of evil: needless breaking: producing brokedness of biological wholes: full functioning times 0

Thus if you have a goal oriented machine, capable of forming a vector sum like picture of the world, you can let it optimise its own functioning toward its own goals by allying with (the health of) the rest of the world, so producing an excellent moral without any compulsions and without any moral to begin with. This is so simple that maybe it could be done with any artificial intelligence capable of counting in order to guide its own actions, i.e. capable of mathematically optimising its actions toward its goals... But of course this does not quarantee that the AI would understand how to safeguard the good health of the world: for that there is the need of a picture of the world and a good understanding. But luckily this seems to show that increasing the intelligence of machines could increase their safety to humans and not the other way around like many are afraid of if they have read stories of robots running wild and ruining the social lives of humans and maybe threatening the whole human kind.InsectIntelligence (talk) 06:23, 18 November 2007 (UTC)