Workflows¶
The SWAT package includes the ability to call CAS actions and process the output in various ways. These range from simple (calling CAS actions as Python methods and getting a dictionary of results back) to complex (invoking CAS actions in multiple sessions and handling server responses from them directly). The various workflows are described in the following sections to give you an idea of how the classes in SWAT interact with each other.
The Easy Way¶
In the most basic form of calling CAS actions, the actions are called directly on the connection object or a CASTable object. When you load a CAS action set (using builtins.loadactionset), the CAS actions in that action set are added to the connection object as Python methods. For example, when the simple action set is loaded, the CAS connection object is extended with methods for summary, freq, crosstab, etc. These can then be called on the CAS connection object directly.
In addition, any CASTable objects that are registered with that connection also get methods for those actions. When you call a CAS action on a CASTable object, the table= (or name= and caslib= in some cases) parameter get populated automatically with that table object before it is sent to the CAS server.
Here is a simple example. We are reading a CSV file using the CAS.read_csv() method. This loads the CSV data into a CAS table and returns a CASTable object. You can then use many of the standard pandas.DataFrame methods such as pandas.DataFrame.head() to interact with the CAS table on the server.
In [1]: import swat
In [2]: conn = swat.CAS(host, port, userid, password)
In [3]: tbl = conn.read_csv('https://raw.githubusercontent.com/'
...: 'sassoftware/sas-viya-programming/master/data/cars.csv')
...:
NOTE: Cloud Analytic Services made the uploaded file available as table TMPDUYYOYH3 in caslib CASUSER(kesmit).
NOTE: The table TMPDUYYOYH3 has been created in caslib CASUSER(kesmit) from binary data uploaded to Cloud Analytic Services.
In [4]: tbl.head()
Out[4]:
Selected Rows from Table TMPDUYYOYH3
Make Model Type Origin ... MPG_Highway Weight Wheelbase Length
0 Acura MDX SUV Asia ... 23.0 4451.0 106.0 189.0
1 Acura RSX Type S 2dr Sedan Asia ... 31.0 2778.0 101.0 172.0
2 Acura TSX 4dr Sedan Asia ... 29.0 3230.0 105.0 183.0
3 Acura TL 4dr Sedan Asia ... 28.0 3575.0 108.0 186.0
4 Acura 3.5 RL 4dr Sedan Asia ... 24.0 3880.0 115.0 197.0
[5 rows x 15 columns]
You can call CAS actions directly on the CAS connection object and pass in the table argument, or you can call the CAS action directly on the CASTable object from CAS.read_csv().
In [5]: conn.summary(table=tbl)
Out[5]:
[Summary]
Descriptive Statistics for TMPDUYYOYH3
Column Min Max N ... TValue ProbT Skewness Kurtosis
0 MSRP 10280.0 192465.0 428.0 ... 34.894059 4.160412e-127 2.798099 13.879206
1 Invoice 9875.0 173560.0 428.0 ... 35.196963 2.684398e-128 2.834740 13.946164
2 EngineSize 1.3 8.3 428.0 ... 59.656105 3.133745e-209 0.708152 0.541944
3 Cylinders 3.0 12.0 426.0 ... 76.913766 1.515569e-251 0.592785 0.440378
4 Horsepower 73.0 500.0 428.0 ... 62.173176 4.185344e-216 0.930331 1.552159
5 MPG_City 10.0 60.0 428.0 ... 79.229235 1.866284e-257 2.782072 15.791147
6 MPG_Highway 12.0 66.0 428.0 ... 96.729204 1.665621e-292 1.252395 6.045611
7 Weight 1850.0 7190.0 428.0 ... 97.526890 5.812547e-294 0.891824 1.688789
8 Wheelbase 89.0 144.0 428.0 ... 269.196577 0.000000e+00 0.962287 2.133649
9 Length 143.0 238.0 428.0 ... 268.525733 0.000000e+00 0.181977 0.614725
[10 rows x 17 columns]
+ Elapsed: 0.0168s, user: 0.016s, sys: 0.004s, mem: 5.38mb
In [6]: tbl.summary()
Out[6]:
[Summary]
Descriptive Statistics for TMPDUYYOYH3
Column Min Max N ... TValue ProbT Skewness Kurtosis
0 MSRP 10280.0 192465.0 428.0 ... 34.894059 4.160412e-127 2.798099 13.879206
1 Invoice 9875.0 173560.0 428.0 ... 35.196963 2.684398e-128 2.834740 13.946164
2 EngineSize 1.3 8.3 428.0 ... 59.656105 3.133745e-209 0.708152 0.541944
3 Cylinders 3.0 12.0 426.0 ... 76.913766 1.515569e-251 0.592785 0.440378
4 Horsepower 73.0 500.0 428.0 ... 62.173176 4.185344e-216 0.930331 1.552159
5 MPG_City 10.0 60.0 428.0 ... 79.229235 1.866284e-257 2.782072 15.791147
6 MPG_Highway 12.0 66.0 428.0 ... 96.729204 1.665621e-292 1.252395 6.045611
7 Weight 1850.0 7190.0 428.0 ... 97.526890 5.812547e-294 0.891824 1.688789
8 Wheelbase 89.0 144.0 428.0 ... 269.196577 0.000000e+00 0.962287 2.133649
9 Length 143.0 238.0 428.0 ... 268.525733 0.000000e+00 0.181977 0.614725
[10 rows x 17 columns]
+ Elapsed: 0.0161s, user: 0.014s, sys: 0.005s, mem: 5.36mb
In [7]: conn.close()
The output from the CAS action methods is always a CASResults object. This is simply an ordered Python dictionary with a few extra attributes and methods added. You can see the keys printed in the output above (surround by square brackets). This output only contains a single key: Summary.
Here is a diagram showing the process of calling and action and consuming the responses into the CASResults object.
Using Response and Result Callbacks¶
The next workflow is to use callback functions to handle either the responses from the CAS server, or the individual result keys in the responses. You still use the CAS action methods on the CAS connection object, but this time you add either a responsefunc= or resultfunc= function argument.
The result callback function takes five arguments: key, value, response, connection, and userdata. Those are the result key and value, the response the result belongs to, the connection that the result belongs to, and an arbitrary user data structure.
The response callback just takes three arguments for the response, connection, and user data structure.
The result callback and response callback are called for each result and response, respectively. Keep in mind that you can only specify one or the other (a response callback would override a result callback). If you want to keep any sort of state information between calls, you can store it in the userdata argument and return it. The returned value will get passed in as the userdata argument on the next call.
Here is an example demonstrating both styles of callbacks.
In [8]: import swat
In [9]: conn = swat.CAS(host, port, userid, password)
In [10]: tbl = conn.read_csv('https://raw.githubusercontent.com/'
....: 'sassoftware/sas-viya-programming/master/data/cars.csv')
....:
NOTE: Cloud Analytic Services made the uploaded file available as table TMPPUWGQUD8 in caslib CASUSER(kesmit).
NOTE: The table TMPPUWGQUD8 has been created in caslib CASUSER(kesmit) from binary data uploaded to Cloud Analytic Services.
In [11]: tbl.head()
Out[11]:
Selected Rows from Table TMPPUWGQUD8
Make Model Type Origin ... MPG_Highway Weight Wheelbase Length
0 Acura MDX SUV Asia ... 23.0 4451.0 106.0 189.0
1 Acura RSX Type S 2dr Sedan Asia ... 31.0 2778.0 101.0 172.0
2 Acura TSX 4dr Sedan Asia ... 29.0 3230.0 105.0 183.0
3 Acura TL 4dr Sedan Asia ... 28.0 3575.0 108.0 186.0
4 Acura 3.5 RL 4dr Sedan Asia ... 24.0 3880.0 115.0 197.0
[5 rows x 15 columns]
In [12]: def result_cb(key, value, response, connection, userdata):
....: print('>>> RESULT', key, value)
....: return userdata
....:
In [13]: tbl.summary(resultfunc=result_cb)
>>> RESULT Summary Descriptive Statistics for TMPPUWGQUD8
Column Min Max N ... TValue ProbT Skewness Kurtosis
0 MSRP 10280.0 192465.0 428.0 ... 34.894059 4.160412e-127 2.798099 13.879206
1 Invoice 9875.0 173560.0 428.0 ... 35.196963 2.684398e-128 2.834740 13.946164
2 EngineSize 1.3 8.3 428.0 ... 59.656105 3.133745e-209 0.708152 0.541944
3 Cylinders 3.0 12.0 426.0 ... 76.913766 1.515569e-251 0.592785 0.440378
4 Horsepower 73.0 500.0 428.0 ... 62.173176 4.185344e-216 0.930331 1.552159
5 MPG_City 10.0 60.0 428.0 ... 79.229235 1.866284e-257 2.782072 15.791147
6 MPG_Highway 12.0 66.0 428.0 ... 96.729204 1.665621e-292 1.252395 6.045611
7 Weight 1850.0 7190.0 428.0 ... 97.526890 5.812547e-294 0.891824 1.688789
8 Wheelbase 89.0 144.0 428.0 ... 269.196577 0.000000e+00 0.962287 2.133649
9 Length 143.0 238.0 428.0 ... 268.525733 0.000000e+00 0.181977 0.614725
[10 rows x 17 columns]
In [14]: def response_cb(response, connection, userdata):
....: for k, v in response:
....: print('>>> RESPONSE', k, v)
....: return userdata
....:
In [15]: tbl.summary(responsefunc=response_cb)
>>> RESPONSE Summary Descriptive Statistics for TMPPUWGQUD8
Column Min Max N ... TValue ProbT Skewness Kurtosis
0 MSRP 10280.0 192465.0 428.0 ... 34.894059 4.160412e-127 2.798099 13.879206
1 Invoice 9875.0 173560.0 428.0 ... 35.196963 2.684398e-128 2.834740 13.946164
2 EngineSize 1.3 8.3 428.0 ... 59.656105 3.133745e-209 0.708152 0.541944
3 Cylinders 3.0 12.0 426.0 ... 76.913766 1.515569e-251 0.592785 0.440378
4 Horsepower 73.0 500.0 428.0 ... 62.173176 4.185344e-216 0.930331 1.552159
5 MPG_City 10.0 60.0 428.0 ... 79.229235 1.866284e-257 2.782072 15.791147
6 MPG_Highway 12.0 66.0 428.0 ... 96.729204 1.665621e-292 1.252395 6.045611
7 Weight 1850.0 7190.0 428.0 ... 97.526890 5.812547e-294 0.891824 1.688789
8 Wheelbase 89.0 144.0 428.0 ... 269.196577 0.000000e+00 0.962287 2.133649
9 Length 143.0 238.0 428.0 ... 268.525733 0.000000e+00 0.181977 0.614725
[10 rows x 17 columns]
In [16]: conn.close()
Here is the flow diagram for using callbacks. Note that only the result or response callback is called. It will not call both.
Handling Multiple Actions Simultaneously¶
The final workflow is the most hands-on, but also offers the most flexibility. For most cases, you can use the callback form to handle any size of data coming back from the CAS server. However, in that mode, you can only run one action at a time. If you want to run simultaneous actions and handle the responses as they come back from either connection, you need to use the CAS.invoke() method.
The CAS.invoke() method calls a CAS action, but doesn’t automatically retrieve the responses. You must iterate over the connection object to get the responses. To iterate over the responses for multiple connections, you can use the getnext() function. Using this technique, you can take advantage of running multiple CAS actions simultaneously while still just running in a single thread on the client.
In [17]: import swat
In [18]: conn1 = swat.CAS(host, port, userid, password)
In [19]: conn2 = swat.CAS(host, port, userid, password)
In [20]: tbl1 = conn1.read_csv('https://raw.githubusercontent.com/'
....: 'sassoftware/sas-viya-programming/master/data/class.csv')
....:
NOTE: Cloud Analytic Services made the uploaded file available as table TMP3VJSANVD in caslib CASUSER(kesmit).
NOTE: The table TMP3VJSANVD has been created in caslib CASUSER(kesmit) from binary data uploaded to Cloud Analytic Services.
In [21]: tbl1.head()
Out[21]:
Selected Rows from Table TMP3VJSANVD
Name Sex Age Height Weight
0 Alfred M 14.0 69.0 112.5
1 Alice F 13.0 56.5 84.0
2 Barbara F 13.0 65.3 98.0
3 Carol F 14.0 62.8 102.5
4 Henry M 14.0 63.5 102.5
In [22]: tbl2 = conn2.read_csv('https://raw.githubusercontent.com/'
....: 'sassoftware/sas-viya-programming/master/data/cars.csv')
....:
NOTE: Cloud Analytic Services made the uploaded file available as table TMPKYYUK6UM in caslib CASUSER(kesmit).
NOTE: The table TMPKYYUK6UM has been created in caslib CASUSER(kesmit) from binary data uploaded to Cloud Analytic Services.
In [23]: tbl2.head()
Out[23]:
Selected Rows from Table TMPKYYUK6UM
Make Model Type Origin ... MPG_Highway Weight Wheelbase Length
0 Acura MDX SUV Asia ... 23.0 4451.0 106.0 189.0
1 Acura RSX Type S 2dr Sedan Asia ... 31.0 2778.0 101.0 172.0
2 Acura TSX 4dr Sedan Asia ... 29.0 3230.0 105.0 183.0
3 Acura TL 4dr Sedan Asia ... 28.0 3575.0 108.0 186.0
4 Acura 3.5 RL 4dr Sedan Asia ... 24.0 3880.0 115.0 197.0
[5 rows x 15 columns]
Now that we have the tables loaded, we can invoke the summary action on each one and retrieve the responses from both connections.
In [24]: tbl1.invoke('summary');
In [25]: tbl2.invoke('summary');
In [26]: for resp, conn in swat.getnext(conn1, conn2):
....: for k, v in resp:
....: print('>>> RESULT', k, v)
....:
>>> RESULT Summary Descriptive Statistics for TMP3VJSANVD
Column Min Max N ... TValue ProbT Skewness Kurtosis
0 Age 11.0 16.0 19.0 ... 38.884748 8.049260e-19 0.063612 -1.110926
1 Height 51.3 72.0 19.0 ... 52.997076 3.202374e-21 -0.259670 -0.138969
2 Weight 50.5 150.0 19.0 ... 19.144897 2.045197e-13 0.183351 0.683365
[3 rows x 17 columns]
>>> RESULT Summary Descriptive Statistics for TMPKYYUK6UM
Column Min Max N ... TValue ProbT Skewness Kurtosis
0 MSRP 10280.0 192465.0 428.0 ... 34.894059 4.160412e-127 2.798099 13.879206
1 Invoice 9875.0 173560.0 428.0 ... 35.196963 2.684398e-128 2.834740 13.946164
2 EngineSize 1.3 8.3 428.0 ... 59.656105 3.133745e-209 0.708152 0.541944
3 Cylinders 3.0 12.0 426.0 ... 76.913766 1.515569e-251 0.592785 0.440378
4 Horsepower 73.0 500.0 428.0 ... 62.173176 4.185344e-216 0.930331 1.552159
5 MPG_City 10.0 60.0 428.0 ... 79.229235 1.866284e-257 2.782072 15.791147
6 MPG_Highway 12.0 66.0 428.0 ... 96.729204 1.665621e-292 1.252395 6.045611
7 Weight 1850.0 7190.0 428.0 ... 97.526890 5.812547e-294 0.891824 1.688789
8 Wheelbase 89.0 144.0 428.0 ... 269.196577 0.000000e+00 0.962287 2.133649
9 Length 143.0 238.0 428.0 ... 268.525733 0.000000e+00 0.181977 0.614725
[10 rows x 17 columns]
In [27]: conn1.close()
In [28]: conn2.close()
The flow diagram for handling multiple connections simultaneously is shown below.
