Skip to content

Add a ppx facility to test expressions #403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 17, 2021
Merged

Conversation

lsylvestre
Copy link
Contributor

@lsylvestre lsylvestre commented Jul 11, 2021

This PR introduces a ppx facility to test expressions.

  1. It embeds compiler-libs/pprintast.cmi into the grading environment. This allows to compose reports quoting well-formed pieces of syntaxe produced by ppx-metaquot (e.g. an expression of type Parsetree.expression builded by [%expr e]).

  2. It implements the ppx extension [%printable e] specified in #379 as a shortcut for Test_lib.printable_fun e (Pprintast.string_of_expression [%expr e]).

  3. It implements the ppx extension [%code e] which build the tuple (Code.(e), Solution.(e), [%expr e]). This may be useful for testing an expression e in the scope of the student code, against the same expression e in the scope of the solution while having access to the syntax tree of e (typically for quoting e in the report).


Here is some examples.

  1. Pprintast + ppx-metaquot

The following grader:

let () =
  Test_lib.set_result @@
  let s = Pprintast.string_of_expression [%expr M.(x + 1)] in
  Report.[ Message ([Text "this expression is well-formed:"; Output s], Informative) ]

produces the following report:

this expression is well-formed:
  let open M in x + 1
  1. [%printable ...]

The following grader:

(* solution.ml *)
let apply f x = f x

(* test.ml *)
open Test_lib

let p_incr = [%printable fun n -> n + 1]
let p_fact = [%printable 
                let rec fact n = 
                  if n = 0 then 1 else n * fact (n - 1) 
                in fact]

let () =
  set_result @@ ast_sanity_check code_ast @@ fun () ->
    test_function_2_against_solution
      [%ty: (int -> int) -> int -> int] "apply" ~gen:0
      [(p_incr, 42); (p_fact, 6)]

produces the following report:

Exercice terminé                          2 pts
-----------------------------------------------
Found apply with compatible type.
Computing apply (fun n -> n + 1) 42
Correct value 43                           1 pt
---
Computing apply (let rec fact n = if n = 0 then 1 else n * (fact (n - 1)) in fact) 6
Correct value 720                          1 pt
  1. [%code ...]

The following grader:

(* solution.ml *)
class counter =
  object
    val mutable c = 0
    method get = c
    method reset = c <- 0
    method inc = c <- c + 1
  end

(* test.ml *)
open Test_lib
open Report

module type S = sig 
  class counter :
    object
      method get : int
      method reset : unit
      method inc : unit
    end
end

let test_expression (e_code, e_solution, e_ast) =
  let descr = [ Text "The expression:"; 
                Output (Pprintast.string_of_expression e_ast) ] in
  let res_code     = exec (fun () -> e_code) in
  let res_solution = exec (fun () -> e_solution) in
  if res_code = res_solution
  then Message(descr @ [Text "seems correct."],Success 1)
  else Message(descr @ [Text "produces an unexpected result."], Failure)
;;

let () =
  set_result @@ ast_sanity_check code_ast @@ fun () ->
  test_student_code [%ty:(module S)] @@ fun code ->
    let module Code = (val code : S) in
    
    (* now, we can safely use Code, since it is compatible with S *)

    [ test_expression [%code let c = new counter in c#get];
      test_expression [%code let c = new counter in 
                             for i = 1 to 10 do c#inc done; 
                             c#get];
      test_expression [%code let c = new counter in 
                             c#inc; c#reset; c#get] ]

produces the following report:

Exercice incomplet                        2 pts
-----------------------------------------------
The expression:                            1 pt
  let c = new counter  in c#get
seems correct.
---
The expression:                            1 pt
  let c = new counter  in for i = 1 to 10 do c#inc done; c#get
seems correct.
---
The expression:                            0 pt 
  let c = new counter  in c#inc; c#reset; c#get
produces an unexpected result.

@lsylvestre lsylvestre changed the title Add a ppx facility for testing expressions Add a ppx facility to test expressions Jul 12, 2021
@erikmd erikmd added the kind: feature New user-facing feature. label Aug 25, 2021
@yurug yurug self-assigned this Sep 15, 2021
@erikmd erikmd added this to the learn-ocaml 0.13 milestone Sep 15, 2021
@lsylvestre
Copy link
Contributor Author

Hi @yurug,

I have just adapted this PR :

  • for compatibility with OCaml 4.12
  • to avoid a conflict with the obsolete file src/grader/build.ocp

Thank you, have a good day.

@yurug
Copy link
Collaborator

yurug commented Sep 17, 2021

Thank you @lsylvestre !

@yurug yurug merged commit 526bc07 into ocaml-sf:master Sep 17, 2021
@lsylvestre
Copy link
Contributor Author

Many thanks @yurug !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: feature New user-facing feature.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants