Part 2 – Creating the best possible user experience for social video apps
In Part 1, we looked at some of the key considerations for building a group live video app for mobile along the lines of Houseparty and Facebook Bonfire, and how the OpenTok platform can provide the solutions to some of the hurdles caused by using WebRTC off-the-shelf. In Part 2, we’ll look at some specific features and code which can be used to create an awesome user experience so your users will fall in love with the app.
Give your users the best possible video experience
Further features to enhance the social experience
Besides the benefits of using OpenTok in the client code over raw WebRTC, there are other benefits in the server side that will make your solution look better to the end user. In Part 1, we talked about the network savings of having the OpenTok server as the distributor of all the media, but having a server in the middle enables it to add things like session recording and archiving, media adaptations and better congestion detection algorithms. It also allows for intelligent quality controls and manageability tools which make it ideal for building a business.
One of the improvements that our servers brings to user experience is simulcast. Simulcast allows a client to send several streams with different qualities, and allows a receiver to receive the best quality based on his network speed or CPU constraints.
If you don’t use simulcast, all clients will receive the same quality stream, and that quality will be determined by the most restricted device in the session.
As social video apps are used in a wide range of settings, this feature means that you are constantly providing the best possible stream quality to all participants, whether they are in a coffee shop, their bedroom or out and about.
Must-dos when using OpenTok for social apps
We have seen a lot of technical concepts and benefits that using OpenTok will bring over using other technologies like raw WebRTC. Many of them are very important if you plan to build a social app with media communication.
We have already seen the benefits of using Simulcast to deliver as high quality a stream as possible to callers. Enabling simulcast is something that is done at server level – you don’t need to change any client code to enable it, however there are several things that you should do at client level.
OpenTok SDK’s Subscriber class has a couple of methods to set the preferred resolution and the preferred framerate. When using Simulcast, calling this will give insights about which stream a particular client wants to receive. For example, if your apps renders the peers video view in a layout that gives just 200 by 200 pixels, it is wise to tell the server to send you the stream with dimensions closest to that. If you receive a 1280 by 720 pixel stream you will be wasting resources since you will render it on a smaller view size. In order to do that you can do something like this:
// Swift code // OTSessionDelegate func session(_ session: OTSession, streamCreated stream: OTStream) { let subscriber = OTSubscriber(stream: stream, delegate: self) subscriber.view.frame = CGRect(x: 0, y: 0, width: 200, height: 200) subscriber.preferredResolution = CGSize(width: 200, height: 200) subscriber.preferredFrameRate = 10 }
You can change the resolution and the framerate as many times you need it, and the server will send you the stream more appropriate for each change.
In order to save resources it is advisable that you set both properties every time the subscriber view size changes. If, for example, your app allows some way to feature a single subscriber, you can increase the resolution when featuring the subscriber.
Another important thing to take into account is to disable the subscriber video if a particular subscriber goes out of the screen. Your app might not want to display all subscribers at the same time in the screen. If that happens, you should make sure that you are calling subscriber.subscribeToVideo = false for that subscriber. Setting the value to false saves the CPU from doing some work since the video will not be processed.
Creating your User Interface for Group Live Video with OpenTok
One of the key elements of building a successful mobile application is to offer an attractive user interface that will engage the users.
When using the OpenTok SDK, you will have every video stream offered like a regular view so you can do whatever you do with the rest of them in the platform. In the case of iOS you will have a View element, so you can apply UIAnimation animations or attach it to any components which lays out views.
One of the best choices is to use an UICollectionView where you can set up your views in many ways since it supports specifying a custom layout for them. If you want to see it in action, a good choice is to take a look at our sample app. This sample is using a custom layout to display all the views of the participants of a session.
In the case of Android, the picture is more or less similar but there are two options when rendering the view. We use OpenGL to render the video streams, and Android offers two ways of drawing it, using GLSurfaceView and using TextureViews. Usually TextureView is more flexible in terms of view layouts, since it behaves like a regular View, however GLSurfaceView is slightly more efficient in terms of rendering. When using OpenTok, you can specify when creating a Session which of the two view backends you want to use, as in the following piece of code:
Session s = new Session(MainActivity.this, APIKEY, SESSION_ID, new Session.SessionOptions() { @Override public boolean useTextureViews() { return true; } });
For displaying the Views, in Android the best option is to use the brand new ConstraintsLayout where you can build a rich and responsible interface.
If you want to have a taste of using ConstraintLayout with OpenTok, you can take a look at our sample app here.
Many of the fastest growing social apps in the world use the OpenTok platform for group live video, such as Houseparty, live.ly and Monkey – apps with a truly global reach. If you’re interested in joining their ranks and building your own social app, keep an eye out for our next post, which will teach you how to use CallKit to send notifications to a phone.
And if you can’t wait that long for the next instalment, check out this post which will teach you how to add Snapchat-like filters to your web application, and have a look at our tutorials in the Developer center which will have you up and running in no time. We love social!
The post Hacking Social Video: Building a Group Live Video App: Part 2 appeared first on TokBox Blog.