SAS Viya 3.5 New Features Updated 10082019
SAS Viya 3.5 New Features Updated 10082019
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U s e O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Overview of the release
• Target release date: November 2019
• Built on SAS Viya 3.5
• Products included:
Supporting themes:
• Integration
• Modernization
• Scalability and Performance
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Common Goals
• SAS Viya 3.5
• Micro-services footprint reduction
• Migration from SAS Viya 3.4 to SAS Viya 3.5
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Platform
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Analytics
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U s e O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Analytics
• Intelligent Automation
• Automated Prediction and What-If Capabilities
• Improved Quick Access recommendation experience in SAS Drive
• Interpretability
• Natural language summaries of approachable analytics (decision tree,
network analysis, path analysis, text topics, forecasting, & the new
automated prediction object)
• Integration
• Visual Analytics SDK support in Copy Link dialog
• SAS Drive browse content experience in the Home Pane
• Open
• Accessibility improvements including alt-text for objects and PDF tagging
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Analytics
• Modernization
• Assign data to blank objects
• Improved image object zero-state
• Ready-to-use Page templates
• Links from Key Value objects
• Improved labeling algorithms for line-based charts(line, forecasting, numeric
series)
• Details table support on Parallel Coordinates
• New application look and feel
• Better organized App Switcher
• Undo/Redo now focuses on object that is acted on
• Scalability and Performance
• Significant UI performance improvements in SAS Drive
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Analytics
• Geo
• Multiple Geo objects are available from the Geographic objects
pane
• Support multiple data layers in Geo Maps
• Display labels and data values on region maps
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Data Management
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U s e O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Data Management
• Intelligent Automation
• Data Discovery with AI-driven transformation Suggestions with code based generated
analytic based transformations (One-hot encoding, Center and scale, Impute missing
values)
• Foundation services for metadata: the Catalog API allows for the discovery, creation, and
integration of metadata – Not Public
• Integration
currently at risk.
• SAS Folder Integration
Remediation is
• CAS and SPRE DS2 DQ Function support ongoing
• DQ microservice will be able to read QKB contents from a Compute Server location
• Data Profiling in Compute Server
• AWS S3 Parquet file support
• Azure Data Lake Storage orc and csv file support
• Salesforce and MongoDB ACCESS engines on Viya (LA) - Read and Write
• Access data from a SAS ESP Window
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Data Management
• Modernization
• Usability enhancements including defer “run on open” action
• Insert, reorder, and remove steps in Data Plans (Data Studio)
• New Data Quality Transformations: Manage Columns, Remove Duplicate
• Simple random partition
• Able to add column labels
• Image file ingestion
• Multi-worksheet Excel file ingestion
• Multi-text file ingestion
• Download to CSV | tab delimited file
• Cross-field clustering (Code Only)
• QKB Field Content definitions Updates (new identities - GIVEN NAME, FAMILY
NAME, and MONTH)
• QKB New Field Content definitions for: UK and Norway
• QKB locales limited support Argentina/Singapore – Japanese locale improvements
(Address)
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Data Management
• Scalability and Performance
• Improvements in handling large columned data sources
• Match and cluster performance improvement
• Cloud Data Exchange agent to agent directory/file movement
• Improvements in Hadoop In-Database, Access to Teradata, Redshift, Postgres,
Impala
• Improvements in CAS FedSQL action
• Improvements in PROC SORT and PROC APPEND
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Analytics
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U s e O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Statistics
• Interpretability
• Show all assessment statistics in an individual model
• Confusion matrix
• Modernization
• Export/save GAM /NPLR models
• Derive predicted values from GAM /NPLR models.
• Quantile binning in decision tree
• User option for default statistic for Model Comparison
• User option to treat 2 level variable as category by default
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Statistics
• Modernization
• PROC GAMSELECT (gam.gsmselect action) for generalized additive
model selection
• PROC NMF (nmf.nmf action) for nonnegative matrix factorization
• PROC SIMSYSTEM (simsystem.simsk) for simulation from systems of
distributions defined by input moments
• PROC SANDWICH, for sparse linear regression with sandwich variance
estimation (pretty much what users currently sometimes use
SURVEYREG for with clustered non-survey data). The associated action
will be undocumented for now.
• Nonlinear.nlmod action that fits a nonlinear model that you provide as a
CMP program
• Elastic net selection in PROC REGSELECT (regression.glm action)
• A factor analytic covariance type for PROC MBC (mbc.mbcfit action)
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Data Mining and Machine Learning
• Intelligent Automation
• Public API for Automated Machine Learning Model Training*,
Model Manager Registration and Retraining
• Automatic Feature Engineering Super Node (Feature Machine)*
• Automatic Modeling Super Node (Model Composer)
• Modernization (new methods / enhancements)
• Batch Reinforcement Learning
• SAS Studio model interpretability tasks and snippets
• Automated Feature Engineering: Interaction detection, shadow
features, ranking
• Tree-based modeling default changes, semi-supervised learning,
SVM, sparse ML new PROC
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Data Mining and Machine Learning
• Integration
• Credit Risk Modeling (ie Credit Scoring for Enterprise Miner in Viya,
including New Interactive Grouping Node) – Basic IGN included with
VDMML
• Interpretability
• Partial Dependency Heatmaps
• Kernel Shap Model
• UI Modernization Improvements
• Standardize numeric formatter across all three Model Studio products
• Upgrade to HTMLCOMMONS 10.0 for Model Studio
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Data Mining and Machine Learning
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Data Mining and Machine Learning
Deep Learning
• Support model training and scoring on GPU for the following
new methods
• Mask R-CNN for instance segmentation
• Deep clustering
• Deep survival analysis
• BERT (with new transformer layer)
• Siamese, triplet, and Quartet networks (with new embedding loss layers)
• Image embedding with angular Softmax loss function
• ShuffleNet V2 (with new split layer)
• New model evaluation metrics for classifications and segmentations
• Data augmentations for object detection and segmentation
• Support CNN layers in RNN models
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Data Mining and Machine Learning
Computer Vision
• Integration
• ESP: field based cropping
• Extract images from PDF files
• Modernization (enhancements)
• Read more than one image into a row (e.g. 1 mask + 1 image)
• New methods in processBiomedicalImages action
- Histogram equalization
- Markov random field
• New methods in processImages action
- Image rotation (any degree)
- Perspective transformation
- Resize image with fixed aspect ratio
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Data Mining and Machine Learning
Speech to Text
• Modernization
• Norwegian language support for STT
• Portable acoustic models (training on CPU, scoring on GPU, and vice
versa)
• Integration
• Long audio file support (DLPY)
• End-to-end STT pipeline in DLPY
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Forecasting
• Modernization
• Configurable forecasting view within the forecasting nodes
• Nested facets in facet panel (Facet Team development, VF integrates)
• Facet panel search uses “contains” (Facet Team development, VF consumes)
• Implement filtering on more columns in Override Management Table
• Optimization on Viewers for more efficient rendering of plots
• New Time Series Dimension Reduction (TDR) Package
- Piecewise Aggregate Approximation (PAA) method
• Open
• New External Language (EXTLANG) Package
- Support for Open Source code - Python and R
- Code based integration; no model specification
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Forecasting
• Scalability and Performance Improvements
• Modeling nodes
• Overrides
• Reduce number of API calls from facet panel (Facet Team development,
VF consumes)
• UI Modernization Improvements
• Standardize numeric formatter across all three Model Studio products
• Upgrade to HTMLCOMMONS 10.0 for Model Studio
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Text Analytics
• Intelligent Automation
• Removed "experimental" label for concepts automatic rule
generation functionality
• Support for all LITI rule types in sandbox tab for concepts node
• Improved auto-complete in rule editor
• Support for new pre-defined concepts in Romanian and Finnish
languages
• Integration
• Support for saving output data from concepts node in VA ready
format
• Performance and scalability
• Improved category rulegen performance using sampling
• Improved performance for empty taxonomies
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Visual Text Analytics
• Backend improvements
• Removed business logic from UI using link filtering
• Updates to project metadata storage
• UI Modernization Improvements
• Standardize numeric formatter across all three Model Studio products
• Upgrade to HTMLCOMMONS 10.0 for Model Studio
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
SAS® Optimization
• Modernization:
• solveBlackbox action: optimize any function; tune any code
• runOptmodel action: 100%+ replacement value vs. PROC OPTMODEL
- BY-group processing
- Distributed execution of COFOR loops
- SUBMIT block: submit any CASL code from within the action
• MILP solver: irreducible infeasible set (IIS) detection
• NLP solver: Interior point direct is the new default algorithm
• Performance improvements: LP, MILP, and NLP solvers
• Enhanced controls:
• Network solver: minimum cut: specify nodes to separate
• NLP solver: additional multistart sampling method, new reformulation option
• Migration: minimal (or zero) changes to SAS/OR® programs
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
SAS® IML
• IML Action Set: contains the iml action
• Enables SAS/IML programs to run in SAS Viya
• Reads and writes data in parallel; supports parallel
computation
• Many SAS/IML programs run with no modification
• Additional capabilities:
• Read and write CAS tables
• Run tasks in parallel
• Score CAS data tables
• Call other CAS actions
• Postprocess results of other actions and procedures
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
SAS® IML
Some SAS/IML features are supported only in SAS/IML,
running in SPRE
(which is included with SAS IML on SAS Viya):
• Base SAS functions not supported by the CAS DATA step
• Statements that read or write SAS data sets or files
• Functions that create graphics
• Deprecated functions
• Functions (a few) that will be upgraded to use modern CAS
features
and surface in future releases of SAS IML
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Econometrics
• Modernization
• Many new features added to SAS Econometrics
• State-independent parameter estimation in the HMM procedure/action set
• Heteroscedasticity Consistent Covariance Matrix Estimation (HCCME) in
the CPANEL procedure and action set
• Heteroscedasticity and Autocorrelation Consistent (HAC) covariance
matrices in the CPANEL procedure and action set
• New SASEMOOD engine to access Moody’s databases
• ASTORE availability in the HMM procedure
• Plots of correlations between variables, and tail dependencies in the
CCOPULA procedure
• Bayesian features in CQLIM and CNTSELECT procedures and action sets
(this is at risk but we knew that going into it)*
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Econometrics
• Integration
• ASTORE availability in the HMM procedure
• Scalability and performance
• Development of new distributed algorithms
• Improved marginal empirical approximation in the CCOPULA procedure
•
Thread load rebalancing for unbalanced BY groups in the SEVSELECT
procedure
• Transition
• Plots of correlations between variables, and tail dependencies
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Decision Management
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U s e O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Model Manager
• SAS Viya 3.5
• Migration from Viya 3.4 to Viya 3.5
• Modernization
• UI and User Experience enhancements
• Updated look & feel of performance reports
• Enhancements to model performance, including the addition of Feature
Contribution Index (FCI)
• Integration
• Job Execution Service (JES) to create performance jobs that can be scheduled
(by environment manager)
• Enhanced Workflow Integration
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Model Manager
• Open
• Easier Python deployment – DS2 wrapper automatically added
with the click of a button.
• Publish Python and R models to run time containers with
embedded binaries and score code files included.
• Promote run time containers to local Docker, AWS Docker and
Amazon EKS (elastic kubernetes service) environments
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Intelligent Decisioning
• SAS Viya 3.5 • Modernization
• Migration from Viya 3.4 to Viya 3.5
• Condition nodes with
• Governance
multiple branches
• Decision diagram and components
documentation • Define data query using SAS
• Decision auditing service Studio’s Data Query Builder
• Copy and delete versions of decision
components
• Score rows of data grids with
• Duplicate variables
sub decision
• Copy rule between rule sets • Validate custom codes
• Update a rule set or a sub-decision used
in decisions
• Browse and edit code nodes
• Copy and duplicate treatment attributes • Edit description of custom
• Updates of lookups without republishing code in decision flow
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Intelligent Decisioning
• Scalability and Performance
• Decision flow level logging
• Node level performance logging
• Reduction of memory requirements for executing decisions
• Reduction of services required for executing decisions
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U se O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .
Questions?
sas.com
C o m p a n y C o n fi d e n ti a l – Fo r In te rn a l U s e O n l y
C o p y ri g h t © S A S In st i tu te I n c . A l l ri g h ts re s e rv e d .