1. Gamify the simulation

The AI you develop in Part 1 enables a virtual country to plan for possible futures, and to quantify the utilities of each of those possible futures and the plans/schedules that lead to them. Each of these futures undoubtedly involves other virtual countries, and that future could only be realized if the other countries did what the self-country’s plan called those other countries to do. But in Part 1, there is no acting on the plans that a country develops. Part 1 is about imagining a future rather than realizing a future, akin to the way that minmax search in chess explores the implications of moves without actually making a move.

In part 2, one direction that you can go is to embed your simulations into game players, perhaps coordinating with other students, to build a game that is played by AIs rather than being played by humans — plans developed in Part 1 are executed in Part 2 and actually change the common, virtual world state.

To create a game of AI players in a virtual world, the AI players must each see the same world state, and when a transform or transfer operator is actually executed by any player’s plan, the single global world state is changed, presumably by an omnipotent game manager.

The players must be able to communicate through a shared protocol. Consider the following as a potential start.

(PROPOSE c_i S S_id {c_k | c_k in S} ) where S is a schedule, S_id is a unique id, c_i and the c_ks are countries, and the c_ks are referenced in one or more operations of the schedule S.

The country c_i, which developed S, submits S to a central “blackboard” that all countries in the world can see, or perhaps only c_ks can see. If a c_k country agrees to the schedule, it communicates this to c_i, again perhaps through a central blackboard.

(AGREE c_k S_id), indicating that c_k agrees to S_id (from which c_i can be determined).

If all c_k countries AGREE, the game manager executes the operators in S, presumably in sequence, updating the world state after each operation.

This is the start of a possible protocol, and its one that could possibly be “gamed” — lots to flesh out here.  If this direction is selected, then the major work would seem to be in defining a game manager that (1) moderates communications between countries through a shared “blackboard” that can be accessed by each country (asynchronously?), and (2)  executes agreed upon plans and changes the world state appropriately.

If the protocol functionality were expanded, of course, the game manager would have to accommodate this, and perhaps have more brains than accounting for world state changes. For example, there might be an ASSERT meta-operator in addition to PROPOSE and AGREE, whereby one country tries to take resources from another country — an attempted unilateral transfer, where the game manager would have to mediate the conflict with stochastic elements (e.g., think of the game of Risk, but where warfare is resource specific?). A paper by Reidl and Bulitko for is relevant to this Part 2 direction.

Mark O. Riedl and Vadim Bulitko (2013). Interactive Narrative:
An Intelligent Systems Approach, AI Magazine, Spring Issue, 67—77. https://www.aaai.org/ojs/index.php/aimagazine/article/view/2449

2. Learning

There are opportunities for various forms of learning in part 2.

Macro learning is one possibility. A macro is a sequence of two or more operators that are conjoined for execution as a package. In principle, a complete grounded schedule can be a macro, but the most useful macros are broadly applicable and often only a few operators long. Typically, in a macro there are variables so the macro can be used for different situations, in our case different countries, different resources, and different quantities of resources. For example, consider a “free-trade” macro which might be only two operators (both transfers) long. One country ?c_i transfers a resource to another country, ?c_k, and then ?c_k transfers another resource to ?c_i in exchange. “Trades” of resources are mutually beneficial and thus would be frequently included in schedules. This macro might very reasonably be part of your Part 1 implementation, but you would “hard-code” the macro in an attempt to reduce search cost. In Part 2, the goal would be to learn this and other macros that proved to be generally helpful.

3. World Modeling

You may significantly expand modeling of the virtual world, perhaps remaining in the confines of Transfer and Transform operators, or perhaps defining additional operators. We have or will discuss modeling recycling and reuse and regeneration, the water cycle, ecosystems, automated updates of resources such a population, limited or all-out war (which need not be part of a gamification involving a game manager), and catastrophes. But other processes can be modeled too, such as financial and economic processes.