Skip to content

Webview support in the Zabaan SDK

This document is a guide for developers to understand the implementation of the Zabaan within webviews.

For a standard Android Activity/Fragment based integration, this document can be ignored.

Overview of steps required by developers

  • Embed getPosition(elementId) into JS code. This function will detect pixel values to show the assistant on and provide it back to the android code in the form of:
{
    "x":"<px val>",
    "y":"<px val>"
}
  • Monitor for changes to the screens using doUpdateVisitedHistory
  • On trigger of the assistant(Use AssistantStateListener to capture event on assistant image), use interaction request to submit to the SDK as depicted in Creating InteractionRequests Section. Identification of the audio file will be a combination of the screen name and the index. This management will need to be done in the Activity that has Webview.
  • Detect if screen change occurs and call Zabaan.getInstance().stopZabaanInteraction(); in the event it is active.

If you are looking to implement this within a WebView, there are certain responsibilities that shift to the app developer. They are listed below:

Screen change triggers

In an Activity based architecture, the SDK can detect screen changes. However, in a webview context, it is the responsibility of the app developer to inform the SDK of a change in screen. 

webView.setWebViewClient(new WebViewClient() {
            @Override
            public void doUpdateVisitedHistory(WebView view, String url,
                              boolean isReload) {
                //Detect when the screen changes and set appropriate state. 
                if (url.contains("<link_for_home_page>")) {
            try {
                        CoordinateInteractionRequest cRequest = new CoordinateInteractionRequest.Builder()
                        .setIndex(index) // Index value to keep track of interaction
                        .setX(x) // X value
                        .setY(y) // Y value
                        .setState("state_for_home")
                        .build();
                Zabaan.getInstance().playInteraction(cRequest);
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }else if(url.contains("<link_for_mobile_payment_page>")){
             try {
                 // Index value to keep track of interaction
                 CoordinateInteractionRequest cRequest = 
                                        new CoordinateInteractionRequest.Builder().setIndex(index) 
                        .setX(x) // X value
                        .setY(y) // Y value
                        .setState("state_for_payment")
                        .build();
                Zabaan.getInstance().playInteraction(cRequest);
                        } catch (Exception e) {
                            e.printStackTrace();
                        }
                                }
                return super.doUpdateVisitedHistory(view, request);
            }
        });

Identifying finger animation positioning

You will need to inject this function into your javascript code.

function getComputedPosition(id) {
    var els = document.querySelectorAll('*[id]')
    for (i = 0; i < els.length; i++) {
      console.log(els[i].id)
    }
    var el = document.getElementById(id);
   const rect = el.getBoundingClientRect();
    var parentElement = document.getElementById(id).parentElement.nodeName;
    const parentRect = el.getBoundingClientRect();

   const scrollLeft = window.pageXOffset ; 
   const scrollTop = window.pageYOffset ; 

   return {
      centreY: parentRect.top + rect.top  + rect.height/2 + scrollTop,
      centreX: rect.left + rect.width / 2 + scrollLeft,
   };
};

Assistant Interaction Listener

You can listen to audio events by implementing AssistantInteractionListener.

To use this listener you can implement this interface in activity class and use Zabaan.getInstance().setAssistantInteractionListener(YourActivity.this) method

This interface provides the following callbacks to you :

  • assistantSpeechDataDownloaded(ZabaanLanguages language)
  • assistantSpeechDataDownloadError()
  • assistantSpeechStart(ZabaanSpeakable interaction)
  • assistantSpeechEnd(ZabaanSpeakable interaction)
  • assistantSpeechError(ZabaanSpeakable interaction, Exception e)

Creating Interaction Requests

The interaction request object allows you to customize the following:

  • setState: We use this mandatory input to pick out the appropriate audio - view id pairing.
  • setFingerAnimation: We use this optional input to override the default finger animation for this specific request.
  • setX: Helps you set the X Axis position of the finger animation.
  • setY: Helps you set the Y Axis position of the finger animation.
  • setIndex: Mandatory to set when using x and y , this index position will be sent back in assistantSpeechEnd or in assistantSpeechError to let developers know which Interaction Request has finished playing or caused the error.

Example code to create CoordinateInteractionRequest and submitting it to Zabaan

CoordinateInteractionRequest cRequest = new CoordinateInteractionRequest.Builder()
                        .setIndex(index) // Index value to keep track of interaction
                        .setX(x) // X value
                        .setY(y) // Y value
                        .setState("state")
                .setFingerAnimation(AnimationClickNormal.INSTANCE)
                        .build();
Zabaan.getInstance().playInteraction(cRequest);

Chaining Interaction requests for screen.

Zabaan provides you with appropriate callbacks for when audio and animations have stopped (i.e. assistantSpeechEnd) or run into error (assistantSpeechError).

The interaction object has a getter getIndex() which will be the same value as one sent in CoordinateInteractionRequest object. By keeping track of Index value, the developers can chain the next CoordinateInteractionRequest.

@Override
    public void assistantSpeechEnd(ZabaanSpeakable interaction) {
        Zabaan.getInstance().stopZabaanInteraction();
        int currentInteractionIndex = interaction.getIndex();
        int newInteractionIndex = interaction.getIndex() + 1;

        if (currentInteractionIndex == 1) {
            if (interaction.getState().equals(<first state>)) {
                                // CREATE INTERACTION REQUEST
            } else if (interaction.getState().equals(<another state>)) {
                                // CREATE INTERACTION REQUEST
            }
        }
    }

For more detailed information about Zabaan features, please refer to the main document below

Zabaan Integration guide

Back to top