Introduction:
Like any website, be it of news or a social networking, where real-time updates are desired and hence get notified automatically without hitting an F5 or page refresh, it isn’t a big ask to expect the same in an Office 365 architecture.
For demonstration, I take an example of an SPFX webpart, which auto refreshes on the page any change done to the underlying list containing its source data. We have used here, webhook as a mechanism to listen to list events and to broadcast the same using socket.io library.
Logic apps could be another option to using webhook for a less complex logic, only if you are aware of its limitations:
- At present, the ‘item deletion’ even cannot be captured.
- Its asynchronous behavior does not guarantee the order of changes performed, which might be of some concern if event timing matters, and I’m pretty sure, in a lot of cases it does .
Hang on until you reach the end of the article where I discuss how to avoid some errors while building the broadcaster in Azure.
What is a webhook?
Webhooks are data and executable commands sent from one app to another over HTTP, instead of through the command line in your computer. They are formatted in XML, JSON, or form-encoded serialization and are so termed as they are software hooks —or functions that run when something happens over the web.
Why webhook as a broadcaster?
Webhooks use HTTP Connection, and hence the connection gets closed once the response is received by the client. But for a real-time application the connection must be alive and the client should get the real time data whenever the backend server/service is updating the data. So, here comes the need of a WebSocket protocol, which must be wrapped over the webhook to achieve this feature in an Office 365 architecture.
WebSocket, being bidirectional communication protocol, can send the data from the client to the server or from the server to the client by reusing the established connection channel. The connection is kept alive until terminated by either the client or the server.
I have divided my solution into 3 parts:
- WebhookBroadCasterPOC: An Azure Node JS web application (Socket IO server application)
- SocketEmitLib: An SPFX library using socket.io.client (reusable client for listening to Azure Node JS web app)
- rxJsEventEmitterlib: An SPFX library to emit events (reusable for broadcasting events using rx-lite)
Let us begin with the BroadCaster Azure Node JS web app.
1.WebhookBroadCasterPOC:
It is a nothing but a simple Azure Web App Service built in Node JS stack.
Here we have used an npm package of Socket. IO ( version 2.0.4) to enable the real-time bidirectional event-based communication It works on every platform, browser or device, focusing equally on reliability and speed. Socket.IO is built on top of the WebSocket API (Client side) and Node.js.
We have used express js to expose the get and post methods, and body-parser to parse the incoming request body.
We have 2 portions of the post method, the first part for the webhook subscription.
And the second part for broadcasting events which is our actual motive. In this part, we are broadcasting the webhook response body over a WebSocket channel using socket.io.
The entire piece of code can be summarized as:
Server.js
Solution Structure:
Index.html
This solution should be hosted as an Azure web app with Windows OS and Node 10 or 12 LTS stack.
2. SocketEmitLib:
This is the second part of my solution, which is nothing but an SPFX library solution built on react which uses socket.io.client npm package to listen our previously created azure webhook broadcaster.
This is the code to connect to the socket URL (here in our case the azure web app (Node JS) in the above step).
Once the socket URL is connected, we are listening to the event “list:changes” which is emitted by the Azure web app broadcaster, and then “getListchanges” method of the SharePoint is called to get the actual item changed (added, updated, or deleted).
this.socket
.on("list:changes", (data) => {
console.log("list:changes");
console.log(JSON.stringify(data));
//debugger;
// this._lastQueryDate = moment();
_objResponse = new ReturnResponse();
let utility = new SPProvider(SiteUrl);
utility
._getListChanges(
data,
SiteUrl,
this._lastQueryDate,
listName
)
.then((changes: any) => {
console.log(JSON.stringify(changes));
if (changes != "") {
// debugger;
changes.forEach((element) => {
let changeType = element.ChangeType;
//insert
if (changeType == 1) {
this.getListItemValues(utility,element,isRenderListAsStream).then((itemArr)=>{
this._lastQueryDate = moment();
_objResponse.AddedItems.push(...itemArr);
this.emitResponse(_objResponse);
resolve(JSON.stringify(_objResponse));
});
} else if (changeType == 3) { // delete
console.log(element);
this._lastQueryDate = moment();
_objResponse.DeleteItems.push(element);
this.emitResponse(_objResponse);
resolve(JSON.stringify(_objResponse));
}
else if (changeType == 2) { //update
// this._lastQueryDate = moment();
debugger;
this.getListItemValues(utility,element,isRenderListAsStream).then((itemArr)=>{
this._lastQueryDate = moment();
_objResponse.UpdatedItems.push(...itemArr);
this.emitResponse(_objResponse);
resolve(JSON.stringify(_objResponse));
});
}
});
}
})
.catch((e) => {
console.log(e);
reject('error');
});
});
3. rxJsEventEmitterlib:
The SocketEmitLib SPFX library solution uses this rxJsEventEmitterlib to emit the data output of the getListChanges call.
When an SPFX webpart uses the SocketEmitLib library, first we need to call the method which establishes a socket connection. Once the control has gone back to the webpart back from the library, we need a subscriber -event connection alive between the library and webpart so that the data received from getListchanges can be sent to the webpart, anytime a list item change is happened. This is where the need of this ‘rxjs’ emitter becomes inevitable.
To get the details of the ‘rxjs’ emitter you can refer this article. Below is my usage
SocketEmitLib:
private emitResponse(_objResponse: ReturnResponse) {
this._eventEmitter.emit("listChanges", JSON.stringify(_objResponse));
}
Now I have created a simple webpart project to use these libraries and see the real time change.
I am using a simple SharePoint list, with the below structure to demonstrate as example:
The web part I have used, simply displays the list content on a grid and will render the current data state of the list automatically, within a few seconds.
Below is my web part code:
import * as React from 'react';
import styles from './SocketClientForList.module.scss';
import { ISocketClientForListProps } from './ISocketClientForListProps';
import { escape } from '@microsoft/sp-lodash-subset';
//import * as myLibrary from 'corporate-library';
import { AgGridReact } from 'ag-grid-react';
import 'ag-grid-community/dist/styles/ag-grid.css';
import 'ag-grid-community/dist/styles/ag-theme-balham.css';
import { sp,IChangeQuery } from "@pnp/sp";
import "@pnp/sp/webs";
import "@pnp/sp/lists";
import "@pnp/sp/items";
import { IRenderListDataParameters } from "@pnp/sp/lists";
import * as RxJsEventEmitterLibrary from 'rx-js-event-emitterlib';
import * as SocketClient from 'socket-emit-lib';
export default class SocketClientForList extends React.Component<ISocketClientForListProps, any> {
private readonly _eventEmitter: RxJsEventEmitterLibrary.RxJsEventEmitter= RxJsEventEmitterLibrary.RxJsEventEmitter.getInstance();
/**
*
*/
private GridColumnArray:any[]= [
{headerName: "Title", field: "Title"},
{headerName: "Subject", field: "Subject"},
{headerName: "From", field: "From"},
{headerName: "To", field: "To"},
{headerName: "SentDate", field: "SentDate"}
]
constructor(props) {
super(props);
sp.setup({
spfxContext:this.props.context
});
this.state={
ReturnJSON:'',
columnDefs: [
{headerName: "Make", field: "make"},
{headerName: "Model", field: "model"},
{headerName: "Price", field: "price"}
],
rowData: [
{make: "Toyota1", model: "Celica", price: 35000},
{make: "Ford1", model: "Mondeo", price: 32000},
{make: "Porsche1", model: "Boxter", price: 72000}
],
Error:''
}
this._eventEmitter.on("listChanges",this.eventsubscriber.bind(this));
}
eventsubscriber =(res:string)=>{
console.log('getting from library');
console.log(res);
let currentData:any[] = this.state.rowData;
let SocketLibResponse = JSON.parse(res);
this.setState({
rowData:[]
});
if(SocketLibResponse.ErrorMsg !=''){
this.setState({
Error:SocketLibResponse.ErrorMsg
});
}else{
if(SocketLibResponse.AddedItems != null && SocketLibResponse.AddedItems.length >0){
currentData.push(...SocketLibResponse.AddedItems);
}
if(SocketLibResponse.UpdatedItems != null && SocketLibResponse.UpdatedItems.length >0){
SocketLibResponse.UpdatedItems.forEach((element)=>{
let item = currentData.filter(p=>p.ID == element.ID);
if(item != null && item != undefined && item.length > 0){
let itemIndex = currentData.indexOf(item[0]);
if(itemIndex != -1)
currentData[itemIndex] = element;
}
});
}
if(SocketLibResponse.DeleteItems != null && SocketLibResponse.DeleteItems.length >0){
SocketLibResponse.DeleteItems.forEach((element)=>{
let item = currentData.filter(p=>p.ID == element.ItemId);
if(item != null && item != undefined && item.length > 0){
let itemIndex = currentData.indexOf(item[0]);
if(itemIndex != -1)
currentData.splice(itemIndex,1);
}
});
}
this.setState({
rowData:currentData
})
}
}
componentDidMount(){
this.getListData(this.props.ListName);
let _getListDataAsStream:boolean=true;
const myInstance = new SocketClient.SocketEmitLibLibrary();
console.log(myInstance.name());
myInstance.getListChangedData(
this.props.context.pageContext.site.absoluteUrl,
this.props.ListName,
this.props.BroadCasterWebhookURL,
_getListDataAsStream
).then((res)=>{
//
let currentData:any[] = this.state.rowData;
let SocketLibResponse = JSON.parse(res);
if(SocketLibResponse.ErrorMsg !=''){
this.setState({
Error:SocketLibResponse.ErrorMsg
});
}
});
}
async getListData(listName){
// setup parameters object
const renderListDataParams: IRenderListDataParameters = {
ViewXml: "<View><RowLimit>1000</RowLimit></View>",
};
const list = sp.web.lists.getByTitle(listName);
// render list data as stream
const Results = await list.usingCaching().renderListDataAsStream(renderListDataParams);
let rowData :any[] =[];
let itemResult = Results.Row;
itemResult.forEach((element) => {
let modElement = {...itemResult};
rowData.push(modElement);
});
// log array of items in response
console.log(Results.Row);
this.setStatewebPart(itemResult);
}
private setStatewebPart(itemResult: any) {
this.setState({
columnDefs: this.GridColumnArray,
rowData: itemResult
});
}
public render(): React.ReactElement<ISocketClientForListProps> {
let {ReturnJSON,Error,rowData}=this.state;
return (
<div className={ styles.socketClientForList }>
<div>
<p className={ styles.description }>{escape(Error)}</p>
</div>
<p>{this.props.ListName}</p>
<div
className="ag-theme-balham"
style={{ height: '200px', width: '100%' }}
>
<AgGridReact
columnDefs={this.state.columnDefs}
rowData={this.state.rowData}>
</AgGridReact>
</div>
</div>
);
}
}
Parting Notes for the Azure Node Js Web App
There are some tips which I would like to mention here.
- Azure Web Apps is available in multiple SKUs, which determine the resources available to your site. This includes the number of allowed WebSocket connections. For more information, see the Web Apps Pricing page.
- If client browsers keep falling back to long polling instead of using WebSocket, it may be because of one of the following.
- For Socket.IO to use WebSockets as the messaging transport, both the server and client must support WebSockets. The default list of transports used by Socket.IO is websocket, htmlfile, xhr-polling, jsonp-polling. You can force it to only use WebSockets by adding the following code to the server.js file in the SocketEmitLib solution, at the following places:
- Before: this.socket = io(websocketurl, { transports: [‘websocket’,’polling’, “flashsocket”, “xhr-polling”] });
- After: this.socket = io(websocketurl, { transports: [‘websocket’] });
- Azure web apps that host Node.js applications, use the web.config file to route incoming requests to the Node.js application. For WebSockets to function correctly with Node.js applications, the web.config must contain the following entry:
- For Socket.IO to use WebSockets as the messaging transport, both the server and client must support WebSockets. The default list of transports used by Socket.IO is websocket, htmlfile, xhr-polling, jsonp-polling. You can force it to only use WebSockets by adding the following code to the server.js file in the SocketEmitLib solution, at the following places:
<webSocket enabled=”false”/>
By doing the default IIS websockets module is disabled which conflicts with Node.js specific WebSocket modules such as Socket.IO. If this line is not present, or is set to true, this may be the reason that the WebSocket transport is not working for your application, and for the handshake error:
WebSocket connection to <URL> failed: Error during WebSocket handshake: Unexpected response code: 521
Hope this article helps you publish real-time changes to your website visitors!
