SEFOR Griffin Neutronic Model for Core I-E

Loading a parallel distributed mesh

For Griffin calculations, a split-mesh configuration was used to optimize parallel computational performance, especially with respect to the memory requirement. The workflow involves configuring the mesh split setup prior to running simulations using Griffin. The Griffin executable is utilized to execute the mesh splitting Listing 1:

Listing 1: Griffin command for splitting the mesh

griffin-opt -i Mesh_Core-I-E_3D_450K.i --split-mesh 1440 --split-file Core-I-E_3D_450K_mesh_split_with_cmfd.cpa.gz

Once the pre-split mesh is created, the workflow transitions to simulation mode by substituting the Exodus file with the split .cpr file and activating the “parallel_type=distributed” option to enable distributed processing during the simulation as illustrated in Listing 2 .

Listing 2: Griffin mesh block for using distributed mesh

[Mesh]
  parallel_type = distributed
  [loader]
    type = FileMeshGenerator
    file = 'Core-I-E_3D_450K_mesh_split_with_cmfd.cpa.gz'
  []
  [coarse_mesh]
    type = GeneratedMeshGenerator
    dim = 3
    nx = 35
    ny = 35
    nz = 34
    xmin = -87.0340
    xmax = 87.0340
    ymin = -87.0340
    ymax = 87.0340
    zmin = 0.0000
    zmax = 165.6018
  []
  [assign_coarse_id]
    type = CoarseMeshExtraElementIDGenerator
    input = 'loader'
    coarse_mesh = 'coarse_mesh'
    extra_element_id_name = coarse_element_id
  []
[]
(sfr/sefor/Core_IE/Core-I-E_ENDF71V2_450K_NA-3_NP-3_NAZ-6_CMFD_1440Tasks_30Nodes_48Cores.i)

Group cross sections

Microscopic cross sections generated from MC-3 were used to describe the homogenized zones including the bottom grid plate, core top (Na_steel), Downcomers (inside or outside the vessel DC_IV/DC_OV), radial reflectors and radial shields. For various fuel assemblies, the materials for each region within the fuel assembly was redefined by specifying the atom density for each isotope and using the microscopic cross sections from the MC-3 homogenized zone. Listing 3 shows the Griffin input to define the materail regions for regions within a standard fuel assembly.

Listing 3: Griffin material block for defining materials in axial segments of the standard fuel assembly with UO2 layers.

[Materials]
  [std_assm_std_fuel]
    type = CoupledFeedbackNeutronicsMaterial
    isotopes = 'pseudo_PU239K pseudo_PU240K pseudo_U235_K pseudo_U238_K pseudo_O16__K '
    densities = '4.08031E-03  3.66406E-04   3.83521E-05   1.73945E-02   4.37142E-02'
    block = '110 111'
    material_id = 1
  []
  [std_assm_void]
    type = VoidNeutronicsMaterial
    block = '112 116'
  []
  [std_assm_ss316]
    type = CoupledFeedbackNeutronicsMaterial
    isotopes = 'pseudo_CR50_K pseudo_CR52_K pseudo_CR53_K pseudo_CR54_K pseudo_FE54_K
                pseudo_FE56_K pseudo_FE57_K pseudo_FE58_K pseudo_MO100K pseudo_MO92_K
                pseudo_MO94_K pseudo_MO95_K pseudo_MO96_K pseudo_MO97_K pseudo_MO98_K
                pseudo_NI58_K pseudo_NI60_K pseudo_NI61_K pseudo_NI62_K pseudo_NI64_K '
    densities = '6.90859E-04  1.33222E-02   1.51071E-03   3.76040E-04   3.41037E-03
                 5.35351E-02  1.23633E-03   1.64532E-04   1.22044E-04   1.88080E-04
                 1.17230E-04  2.01769E-04   2.11398E-04   1.21032E-04   3.05815E-04
                 7.61491E-03  2.93327E-03   1.27504E-04   4.06546E-04   1.03540E-04'
    block = '113 117'
    material_id = 1
  []
  [std_assm_beo]
    type = CoupledFeedbackNeutronicsMaterial
    isotopes = 'pseudo_BE9__K pseudo_O16__K '
    densities = '6.84143E-02  6.84143E-02'
    block = '114 115'
    material_id = 1
  []
  [std_assm_ss304]
    type = CoupledFeedbackNeutronicsMaterial
    isotopes = 'pseudo_CR50_K pseudo_CR52_K pseudo_CR53_K pseudo_CR54_K pseudo_FE54_K
                pseudo_FE56_K pseudo_FE57_K pseudo_FE58_K pseudo_NI58_K pseudo_NI60_K
                pseudo_NI61_K pseudo_NI62_K pseudo_NI64_K '
    densities = '8.17351E-04  1.57623E-02   1.78728E-03   4.44885E-04   3.48075E-03
                 5.46400E-02  1.26184E-03   1.67927E-04   6.80694E-03   2.62205E-03
                 1.13973E-04  3.63413E-04   9.25497E-05'
    block = '119 120 122'
    material_id = 1
  []
  [std_assm_na]
    type = CoupledFeedbackNeutronicsMaterial
    isotopes = 'pseudo_NA23_K '
    densities = '2.39160E-02'
    block = '118 121 123'
    material_id = 1
  []
(sfr/sefor/Core_IE/Core-I-E_ENDF71V2_450K_NA-3_NP-3_NAZ-6_CMFD_1440Tasks_30Nodes_48Cores.i)

Solver options

In this VTB model, Griffin simulations utilized the DFEM-SN transport solver. Angular discretization was achieved using Gauss-Chebyshev quadrature with three polar angles and six azimuthal angles. The finite element shape functions for the angular fluxes (primal variables) were first-order MONOMIAL and the maximum scattering anisotropy order was set to NA=3 as illustrated in Listing 4:

Listing 4: Griffin transport block for modeling SEFOR Core I-E

[TransportSystems]
  particle = neutron
  equation_type = eigenvalue
  G = 33
  VacuumBoundary = '10000 20000 30000'

  [sn]
    scheme = DFEM-SN
    family = MONOMIAL
    order = FIRST
    AQtype = Gauss-Chebyshev
    NPolar = 3
    NAzmthl = 6
    NA = 3
    sweep_type = asynchronous_parallel_sweeper
    using_array_variable = true
    hide_angular_flux = true
    collapse_scattering = true
    assemble_fission_jacobian = false
    flux_moment_primal_variable = true
  []
[]
(sfr/sefor/Core_IE/Core-I-E_ENDF71V2_450K_NA-3_NP-3_NAZ-6_CMFD_1440Tasks_30Nodes_48Cores.i)

To enhance computational efficiency, the “using_array_variable” option was enabled, allowing angular fluxes for each group to be stored via MOOSE’s ArrayVariable system. This approach reduces the number of computational kernels and minimizes overall computational cost. Additionally, the “collapse_scattering” option was activated, enabling in-group scattering sources to be directly formed within the scattering kernels, further reducing simulation runtime.

For the cases with CMFD acceleration as shown in Listing 5, the “prolongation_type” was set to multiplicative. This configuration updates the fine mesh flux using the coarse mesh solution in a multiplicative manner, influencing convergence behavior and allowing for performance tuning. A maximum diffusion coefficient of 1.0 was specified using the “max_diffusion_coefficient” parameter. The Richardson outer iteration was configured with an absolute tolerance of 1.0E-6, a relative tolerance of 1.0E-6, and a maximum of 1000 iterations. The inner linear solver used GMRES, with a maximum of 1000 iterations per solve.

Listing 5: Griffin executioner block for modeling SEFOR Core I-E

[Executioner]
  type = SweepUpdate

  richardson_abs_tol = 1e-6
  richardson_rel_tol = 1e-20
  richardson_value = eigenvalue
  richardson_max_its = 1000

  inner_solve_type = GMRes
  max_inner_its = 1000

  cmfd_acceleration = true
  coarse_element_id = coarse_element_id
  prolongation_type = multiplicative
  diffusion_eigen_solver_type = newton
  diffusion_prec_type = lu

  abs_eig_tol = 1e-6
[]
(sfr/sefor/Core_IE/Core-I-E_ENDF71V2_450K_NA-3_NP-3_NAZ-6_CMFD_1440Tasks_30Nodes_48Cores.i)

Griffin simulations were completed after approximately 7 hours using 2016 CPUs distributed over 84 computational nodes from the INL HPC cluster as of the date, with 24 cores per node and collectively total about 16 TB of memory. Subsequent optimization indicated that Griffin’s computational memory requirements could be substantially reduced by activating the 'flux_moment_primal_variable' parameter within the transport solver, which stores flux moments rather than the full angular flux information. The effect of implementing this option is siginificant. Activation of this parameter is strongly recommended for modeling SEFOR.