Building a Stage

To find the maximum triangle, note that one of the vertices will always lie on the boundaries of some arc. This is true because if all vertices do not lie on the boundaries of the arcs, we can "rotate" the triangle until one of the vertices lies on the boundary.

Let's fix one vertex; the maximum triangle that can be inscribed in the circle is equilateral. If it can be inscribed under the current constraints, it will be the answer.

Now suppose the answer gives a triangle with exactly two points lying on the boundary; to maximize the area, the third point should lie at the midpoint of the arc formed by these two points. If such a point does not exist, then the answer will be the triangle where all three points lie on the boundaries.

We will show that if a triangle has only one vertex on the boundary, it must necessarily be an equilateral triangle. If not, we can definitely move one of the points to increase the area of our triangle (this cannot be done only if the triangle is equilateral).

Thus, the answer is found by checking three cases: when one vertex lies on the boundaries and the triangle is equilateral, when exactly two vertices lie on the boundaries, and when all three vertices lie on the boundaries.