First off: Happy new year folks! Last month I decided to make a switch to Phaser 3, which is very close to being released. One of my favorite features includes the new Camera system. I am still doing research in implementing AI for other aircraft and Reinforcement learning is one of the approaches I considered and experimented with during the holidays. Extremely fun, but difficult to harness.
Also note that I’ve recently added a subscribe form at the bottom of the log, so for those of you who wish to receive these dev logs regularly via e-mail, please don’t forget to subscribe! If you are interested in collaborating in the development, please feel free to contact me. Developer support would be great.
All updates are found in the changelog.
Phaser 3
As a close follower of Phaser development, I regularly read through the dev logs. Phaser 3 is currently approaching its first release and beta releases have been available for quite a while. I was hesitant on whether I should stick to Phaser 2 or move the codebase to Phaser 3. Phaser 2 by itself was a great version to work with, but I was eager to explore new features of Phaser 3 and I just could not ignore the already available beta. Of the many new features, the most interesting for this project:
- New camera system, including multi camera support.
- New and improved renderer.
- New Audio system, including pitch tuning.
- Lighting support.
The shift towards Phaser 3 is also a learning step to familiarize with the new API. So in the end it was a well made descision.
Cameras
The new Camera system adds Aa whole new level of sophistication to the game. As I might have mentioned earlier, there is A simulated world, which is currently bounded in width and height, but I am still considering removing the horizontal bound. The world can now be explored via mouse movement while holding the R key, which tells the camera to detach from its main target.
With the addition of multi-camera support, I can consider adding minimaps and target/objective displays, object tracking with lerp motion etc.
Reinforcement learning
This was another experiment that kept me busy and probably even distracted from the main work needed to be done. Reinforcement learning is one way to implement an AI, but definitely not the easy way. I came up with the idea after finding out about reinforcement learning being applied to the flappy bird game. In my attempts, I took the very first JS library I could find, Reinforce JS, analysed the examples and tried to fit them to the aircraft by setting up A simple training objective. The objective was to fly in formation by maintaining horizontal speed. The autopilot was set to the proper altitude, so the agent (the AI) could only act by adjusting throttle.
Mission
Progress is demonstrated.
Briefing
Initial altitude 40.000ft, speed 500KTS. The flight leader is flying with you, so try to keep up with your leader.The autopilot is set to ALT.
Flight
A day night cycle/ sky is added , high altitude contrails and smoke trails, which still need some tweaking and sonic boom. The camera can look freely using the mouse pointer while holding the R key. Also notice how the HUD tilts at the very end of the video, simulating pilot pilot head movement due to exposed g-forces.
You can also see the RL agent in action. The agent observes the environment (leader position, my position, my speed , leader speed), takes some, possibly random , predefined actions (throttle up, throttle down, speed brake full, do nothing) and based upon some pre-defined reward scheme (your too far behind/past the leader, reward = -distance), it should learn to take the actions that minimizes the reward (if im next to the leader, reward = 0, hmmm this looks,nice, try to keep on doing these kind of actions).
In my attempt, there where all sorts of problems I was faced with: Validating if it does learn and if it learns to converge to the right solution, the training time required, the dimension of my state space and the highly non-linear environment. In some test runs the agent was able to maintain perfect formation for about 10 seconds, but afterwards it would diverge. Yet I still believe with a bit more research it should be possible to get some sensible result, but darn you time….
What’s next
With these features added I can work on scenery design, for example some enemy ground objects, Airbases to take-off and land from, buildings and a background environment. Weapon systems, combat flight, AI, mission building would follow afterwards. The UI will be considered in the final stages. The main challenge is the generation of the scenery/world. It might be necessary to implement A mission/scenery editor to facilitate the above. My hands are A bit tied at the moment.
I could definitely use some help to work on these features. In particular developing a mission and scenery builder/editor, which generates the JSON data fed to the simulator via GUI. This could be some standalone application, but a built-in mission/scene editor would be ideal. A JSON Loader for loading aircraft data is implemented, but a generic object loader would be needed. The current JSON data structure of simulated objects might have to be refactored aswell. Data persistance is not yet implemented. Implementing AI for vehicles and aircraft would be another challenge. If you are interested in collaborating, please feel free to contact me.
Scramble JS uses Phaser 3 as game engine. Fore more info, visit Phaser.io.