I received a notice from RESNET that the required HERS Rater Simulation Test has been delayed. The system is up and running for Raters to practice on the simulation; however, RESNET does not want tests to be performed until more issues are resolved. The two notices I received also stated that the September 1, 2017 deadline will also be extended. No word yet on the new deadline date; I'm sure we won't have that date until the simulation tool and the new RESNET test portal and the updated browsers are playing nice with each other and with end users.
So, what's the issue and why was the tool released with problems? There are times that the simulated manometers don't show up. There are times that users experience an error message stating that they are "out of memory". There is confusion about how to access to practice versus taking the test. There is some confusion among training providers regarding correct procedures and answers within the tool. There are other minor issues that are being reported.
The major problems that are causing the testing delay deal mostly with configuration and stress test situations. The system was tested, in what those of us who have worked in software testing call "unit test" mode. This is when an individual component of a system is tested to make sure it works correctly, such as teleporting to different areas and making sure the user ends up where they are supposed to be instead of, as was noted during testing, getting stuck in the floor between simulated levels of the house. Then the simulation was "system tested", where entire scenarios were tested end-to-end to make sure users could get completely through and achieve a 100% correct score. However, budget and time constraints meant that not every practice scenario was 100% system tested and no integration or stress testing was performed.
Integration testing is when the system is tested along with other systems that it is connected to. In this case, that would be testing the simulation tool from within the new RESNET test portal. The portal implementation was delayed, so that testing could not be performed prior to roll-out. Stress testing is when the system is placed under the stress of having lots of users banging on it at the same time or lots of processing happening at once. The team testing the system was not large and were all connected to the Internet from separate connections. Once training providers started to have a classroom of students attempting to take the test at the same time over the same wireless network or Internet access point on computers and servers that were likely not as robust as what the software providers were using; new issues started to pop up. Combine that with the fact that the browsers are still working out the bugs in WebGL and the whole thing is like trying to hit a moving target.
So, what is WebGL? Well, the reason that the first simulation tool had to be scrapped is because it was using a piece of code to manage the virtual reality world being used for the simulation. That piece of code was called the "Unity Player" which had to be downloaded to allow the simulation to run in the browser. The browsers dropped support of that Unity Player plugin, which meant that the simulation tool would no longer run, and they instead incorporated its successor, WebGL (Web Graphics Library) for rendering 3D images in the browser. Experience now shows that the use of WebGL eats up a lot of memory and if the user doesn't have a lot to start with, then the system comes crashing down. Chrome, Firefox and Edge are all still working through bugs in their implementations of WebGL. They dropped one working 3D option and really did not have a fully working replacement.
Both Interplay Energy and RESNET are working hard to address the issues they can. You will notice that the new simulated house does not have as much furniture, doesn't have bushes and trees, and some graphics may look a bit fuzzy. That is all an attempt to remove as many memory-eating extras as possible. You will also see a memory meter at the top right of the screen so you will know if your computer may not be the best choice to use for your testing system. RESNET is working to make their portal more stable and user-friendly. Both companies are working together to ensure training providers understand the hardware and connection requirements behind assuring a good testing experience for HERS Raters. And training providers are using the tools, sending feedback, studying the user guide and discussing the specifics of the scenarios. There's nothing any of us can do about the browsers and WebGL except keep our fingers crossed that they will work out their bugs quickly.
What I find interesting is that the issues are all hardware, browser, capacity, machine type of problems - not issues with the RESNET Standards or HERS Rating processes and procedures. This means that there is nothing keeping all of us from logging into the portal and practicing on the simulation tool one-on-one, which seems to be working pretty well. Go ahead and pull up and read all the help docs and practice with the 2 houses and 12 scenarios available. You have just been given the gift of more time to practice and master this requirement. Now go out there and play!