How to parse website data in android application?

Question

Android

Hello everyone, I'm starting to develop for android and wondered. How to scrape data from website in android app?
After some googling, I wrote this code

public class MainActivity extends Activity {
 
    /** Called when the activity is first created. */
    Button butTest;
    TextView textView;
    String title;
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        butTest = (Button)findViewById(R.id.button);
        textView = (TextView)findViewById(R.id.textView);
        new MyParser().execute("http://beastinvest.su/");
        butTest.setOnClickListener(butOncl);
    }
 
    private View.OnClickListener butOncl = new View.OnClickListener() {
        public void onClick(View v) {
            onPostExecute(title);
        }
    };
    public class MyParser extends AsyncTask<String, Void, String> {
 
        @Override
        protected String doInBackground(String... links) {
            Document doc = null;
 
            try {
                doc = Jsoup.connect(links[0]).get();
                title = doc.title();
            } catch (Exception e) {
                e.printStackTrace();
            }
            return null;
        }
 
    }
    public void onPostExecute(String result) {
        textView.setText(title);
    }
 
}

But he refused to work. Later, one kind person said that the site has DDoS protection - the request unfolds "To visit this site, you need to support javascript and cookies in your browser."
The same person suggested making a parser through webview

MAIN_ACTIVITY

package com.admin.beastinvest;

import android.os.Bundle;
import android.app.Activity;
import android.view.Menu;
import android.view.View;
import android.widget.TextView;

/**
 *
 * Launcher activity.
 *
 */
public class MainActivity extends Activity implements LoadListener {

    private static final String LOAD_URL = "http://beastinvest.su";
    private static final String LOAD_URL_MATCHER = "http://.*beastinvest.su.*";

    private TitleLoader loader;
    private TextView text;

    /**
     * {@inheritDoc}
     */
    @Override
    protected void onCreate(final Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        getActionBar().hide();
        loader = new TitleLoader(this);
        text = (TextView) findViewById(R.id.text);
    }

    /**
     * Invoked on button pressed.
     * @param view
     *          Button
     */
    public void load(final View view) {
        loader.getTitle(LOAD_URL, LOAD_URL_MATCHER, this);
    }

    /**
     * {@inheritDoc}
     */
    @Override
    public boolean onCreateOptionsMenu(final Menu menu) {
        getMenuInflater().inflate(R.menu.main, menu);
        return true;
    }

    /**
     * On title recieved.
     */
    @Override
    public void onLoad(final String title) {
        runOnUiThread(new Runnable() {

            @Override
            public void run() {
                text.setText(title);
            }
        });
    }


}


TITLE LOADER


package com.admin.beastinvest;


import android.content.Context;
import android.webkit.WebChromeClient;
import android.webkit.WebView;
import android.webkit.WebViewClient;
import android.webkit.WebSettings.PluginState;

/**
 *
 * Class to get title of web page through webview.
 *
 */
@SuppressWarnings("deprecation")
public class TitleLoader {

    private final WebView view;
    private String urlMatch;

    /**
     * Constructor.
     * @param context
     *          Context to create webView
     */
    public TitleLoader(final Context context) {
        view =  new WebView(context);
        view.setWebChromeClient(new WebChromeClient());
        view.getSettings().setPluginState(PluginState.ON);
        view.getSettings().setJavaScriptEnabled(true);
        view.addJavascriptInterface(this, "HTMLOUT");
        view.setWebViewClient(new WebViewClient() {

            @Override
            public void onPageFinished(final WebView view, final String url) {
                super.onPageFinished(view, url);
                if (url.matches(urlMatch)) {
                    view.loadUrl("javascript:window.HTMLOUT.onLoad(document.getElementsByTagName('title')[0].innerHTML);");
                }
            }

        });
    }

    /**
     * Method to invoke getting title.
     * @param url
     *          url to load
     * @param urlMatch
     *          filter for url (".*" - if no filter required)
     * @param listener
     *          title receive listener
     */
    public void getTitle(final String url, final String urlMatch, final LoadListener listener) {
        this.urlMatch = urlMatch;
        view.addJavascriptInterface(listener, "HTMLOUT");
        view.loadUrl(url);
    }
}


LOAD LISTENER
package com.admin.beastinvest;

/**
 * Created by Admin on 13.07.14.
 */
public interface LoadListener {

    /**
     * Invoked on title loaded.
     * @param title
     *          page's title
     */
    void onLoad(final String title);
}

The app works on his device, but not on mine. Perhaps this is due to Android L.
How can I parse data from the site?

Answer 1

Yes, there are enough examples on the official site . There is also JSoup . I liked him more.