Manual testing of Release 2022.12.1 Part 3

[alisevych]

Initial set-up

Add link to Plugin build #***

• Sarif report + Problems view
• Cancellation of generation (with Generate / Generate & Run)
• Java docs styles
• Sandboxing
• Logging
• Hanging test timeout, check with Fuzzing/Symbolic
• Overflows, check Summaries
• Context estimator, check nightly runs
• Bugs list

[alisevych]

Security testing (sandboxing)

Basic security

• Project with JDK 8 + manual scenario from PR Add SecurityManager support to block suspicious code #622 #625 + default settings
• Fuzzing only
• Symbolic execution only
• Project with JDK 11 + default settings
• Fuzzing only
• Symbolic execution only
• Concrete executor is invoked from Symbolic execution
• Project with JDK 17 + default settings

With permissions

• add PropertyPermission in ~.utbot\sandbox.policy - check test for property is not in sandbox, execute
• add "permission java.io.FilePermission "<>", "read";" - check test for read not in sandbox, execute
• add AllPermission in ~.utbot\sandbox.policy - check NO tests are in sandbox, try to execute (carefully!!!)
• corrupt permission - check how UTBot fails - try scenario from Too many permissions file rereads - can slow down much #796
• empty file
• remove ~.utbot\sandbox.policy file - check generated tests are in sandbox

Test rendering

• JUnit4
• JUnit5
• TestNG
• JUnit5 with parametrization
• TestNG with parametrization

Code structures

• in private method
• bug to check: Implicit unpermitted operations should be sandboxed #791

[alisevych]

Reproducing:

• Socket.connect and Files.readAllBytes produce errors in Concrete execution #1314
• Unused import of java.net.UnknownHostException #1999