>>296430It's obvious that the vertexes of the triangle have to be on the edge of the circle.
Now, if you pick any 2 random points on the edge of the circle, the triangle will have the largest area if the 3rd vertex is on the other side of the circle, directly in the middle between the other 2 vertexes. This means that the triangle you are looking for is an isosceles triangle.
Can you figure out the rest? You have to use calculus and stuff, can't prove this geometrically alone (I think).