Test matrix

From BOINC


For each release, please do

  1. The general tests.
  2. As many of the other test groups as possible.

If you have experienced problems with BOINC that are not exercised by any of these tests, please post to the boinc_alpha@ssl.berkeley.edu email list.

Windows installer Tests

Windows Compatibility/Private Install

This install mode is selected by unchecking the 'Protected application execution' checkbox and checking the 'All users can control BOINC' checkbox on the advanced configuration screen.

  • Launch BOINC Manager and verify that the client runs and you can control it.
  • Launch Task Manager (CTRL-SHIFT-ESC), switch to the 'Processes' tab and verify that boinc.exe, boincmgr.exe, and any science applications are running as the current user account.
  • Log on as a different user and launch BOINC Manager. A dialog should appear telling you to contact an administrator and have them add you to the 'boinc_users' group.

Windows Secure/Private Install

This install mode is selected by checking the 'Protected application execution' checkbox and unchecking the 'All users can control BOINC' checkbox on the advanced configuration screen.

  • Launch BOINC Manager and verify that the client runs and you can control it.
  • Launch Task Manager (CTRL-SHIFT-ESC), switch to the 'Processes' tab and verify:
    • boinc.exe is running as 'boinc_master'.
    • boincmgr.exe is running as the current user.
    • science applications are running as 'boinc_project'.
  • Log on as a different user and launch BOINC Manager. A dialog should appear telling you to contact an administrator and have them add you to the 'boinc_users' group.

Windows Secure/Public Install

This install mode is selected by checking the 'Protected application execution' checkbox and checking the 'All users can control BOINC' checkbox on the advanced configuration screen.

  • Launch BOINC Manager and verify that the client runs and you can control it.
  • Launch Task Manager (CTRL-SHIFT-ESC), switch to the 'Processes' tab and verify:
    • boinc.exe is running as 'boinc_master'.
    • boincmgr.exe is running as the current user.
    • science applications are running as 'boinc_project'.
  • Log on as a different user and launch BOINC Manager. You should be able to attach to projects, suspend/resume work, etc.

Windows Data File Migration Tests

During installation, choose a new data directory to migrate BOINC's data files to.

After installation check to see that your data files were moved to the new directory and that both BOINC and BOINC Manager launch and can use the data files.

Uninstall BOINC and make sure the data files were moved back to the original directory.

General tests

  • Note fraction done of existing results. Install the new version on top of an older version. Make sure BOINC client and manager start up. Make sure results resume with same fraction done.
  • Make sure graphics work (for all apps that support graphics).
  • Make sure CPU time and fraction done are increasing for running apps.
  • Check all tabs in BOINC manager, make sure data is there.
  • Detach from a project (e.g., alpha test) and reattach. Make sure new work gets downloaded.
  • Let BOINC run long enough to complete a result. Verify that output file is uploaded, that the result is reported, and that new work is downloaded and started.
  • Suspend/resume features: suspend and resume results, file transfers, and projects.
  • Test activity modes 'suspended', 'based on preferences', 'always'.
  • Shut down your computer and reboot it. Make sure client and manager start up OK.
  • Suspend and/or hibernate your computer, and resume it. Make sure client and manager resume OK.
  • If anything seems significantly slower as a result of BOINC, report it as a minor bug.
  • Verify the 'Help' menu items bring up a web browser and loads a web page on the BOINC site.
  • Windows Only: Verify that pressing 'F1' brings up a web browser and loads a web page on the BOINC site.

Clean install

This test may cause you lose work in progress; do it only if you know what you're doing.

  • Remove BOINC (Windows: go control panel -> add/remove programs and uninstall BOINC). Then rename the BOINC directory to a temporary name.
  • Install new BOINC.
  • Verify that client and manager run, and manager brings up the Attach Project wizard.
  • Attach to SETI@home or other projects, and make sure work is downloaded and started.
  • From the time of installation to the time that work is begun there should be no long delays, errors, confusing messages, or anything else that might bother a naive, non-technical computer owner. If there is, report it as a a bug.
  • Detach from SETI@Home or other projects.
  • Exit BOINC, delete the BOINC directory, and rename temporary directory back to BOINC.
  • Reinstall BOINC.

File transfer restart

  • Exit BOINC in the middle of a long file download. Note the fraction done. Restart BOINC. Verify that download resumes from the same point. Use the BOINC Alpha project for this.
  • Same, but disconnect your network cable in the middle of the transfer.
  • Same as above two, but upload instead of download.

Preferences tests

  • Test 'don't run when user active' preference.
  • Test 'leave applications in memory' preference.
  • Test scheduling period preference.
  • Test #processors preference.
  • Test 'use network between hours' preference.
  • Test 'do work between hours' preference.
  • Test 'CPU usage limit' preference.
  • Test RAM usage limit preferences (both of them).
  • Test venue mechanism (e.g., create preferences for 'Home', change your computer's location to 'Home', make sure it gets the right preferences).
  • Update a second project with same account email address, and make sure preferences are propagated to the second project.

These tests can be done by changing preferences on web site, then Updating the project in the BOINC manager.

Tests for modem-connected computers

  • Do general tests from a modem, ISDN, or VPN-connected computer. see the Dial-up Connections page for details about dial-up connections.
  • Test the connection options:
    • set default connection.
    • verify the manager uses the default connection.
    • proper notification when default hasn't been set.
  • Test 'confirm before connect' preference.
    • 'confirm' not set:
      • Test connections are automatically made when 'confirm' isn't set.
      • Test proper operation when no user is logged on (service install).
    • 'confirm' set:
      • Check proper operation with both 'yes' and 'no' replies to the confirmation dialog
      • Check proper operation when confirm dialog is ignored dialog should disappear after a. few minutes, with another prompt later (fill in times).
      • With manager running only in the systray, check balloon popup indicating BOINC needs a network connection.
  • Test 'Disconnect when done' preference.
  • Check that network activity is retried when dialup connection is established - pending scheduler requests and file transfers get retried.
  • Do these tests with both single-user and Service-mode install (Windows).

Tests for computers with personal firewalls

  • Do general tests on a computer that uses a personal firewall (ZoneAlarm, Symantec, Windows XP, etc.). In your test report Comments field, indicate the type of personal firewall.

Tests for computers connected by a proxy

  • Do general tests on a computer connected via an HTTP proxy. Use authentication if possible.
  • Do general tests on a computer connected via a SOCKS proxy. Use authentication if possible.

Tests for laptops

  • Test 'work while running on batteries' preference.

Screensaver tests (Windows, Mac)

  • Test normal case (enter/leave screensaver mode).
  • Open graphics windows from Manager before enter/leave screensaver; make sure they reappear.
  • Graphics change on CPU reschedule (set scheduling period to 1 min).
  • Test 'no graphics capable apps'.
  • Test 'blank screen after X minutes'.
  • Test power-saver modes, e.g. 'turn off monitor after N minutes'.
  • Test password-after-resume options.
  • Test screensaver properties "Preview" function.
  • (Windows only?) Test multi-user – switch to different user and run screensaver.

BOINC Manager Attach Project Wizard

  • Test normal case, invalid URL, valid but non-BOINC URL, projects that are down.
  • Test bad password, bad username.

GUI RPC authentication

  • Try to connect to core client with bad password, from host not on list, etc.

Account manager functions

  • Attach to and detach from an account manager such as GridRepublic or BOINCStats.
  • Verify that the core client periodically synchronizes with the account manager.
  • Verify that the core client won't let you detach from a project that you attached via the account manager.

NOTE: If you find bugs in the account manager itself, report them directly to the account manager admins, not to BOINC Alpha.

Trickle message and intermediate file uploads

  • Let CPDN run long enough so that it does a trickle-up request and an intermediate file upload. Make sure that these succeed.

Multi-user

to be done